Amazon Abandons AI Recruiting Tool After Discovering Anti-Woman Bias
Amazon scraps a secret AI hiring engine after discovering it systematically penalizes female job applicants. The system learns to prefer male candidates by training on a decade of male-dominated resumes.
Amazon abandons a secret artificial intelligence recruiting tool after discovering that the system discriminates against women. The company builds the experimental engine starting in 2014 to automate the review of job applicant resumes and rate candidates on a one-to-five star scale. However, by 2015, the team realizes the AI does not evaluate candidates for technical roles in a gender-neutral way.
The bias stems from the data used to train the machine learning models, which consists of resumes submitted to Amazon over a 10-year period. Because the tech industry is predominantly male, most of these resumes come from men, causing the system to teach itself that male candidates are preferable. The AI actively penalizes resumes that include the word "women's" and downgrades graduates of all-women's colleges.
Although Amazon attempts to edit the algorithms to make them neutral to specific gendered terms, executives ultimately lose hope that the tool can be made fair. The company disbands the project by early 2017 because there is no guarantee that the machine will not find other discriminatory ways to sort candidates. This situation highlights the significant limitations and risks of using machine learning for hiring decisions.