Amazon Scraps Biased AI Recruitment Tool That Penalized Women
Amazon discontinues an internal AI hiring tool after discovering it systematically discriminates against female job applicants due to flawed training data.
Amazon abandons an experimental AI recruitment tool after discovering it exhibits a strong bias against female job applicants. The company builds the system in 2014 to automate the review of incoming resumes, hoping to create an engine that consistently identifies top talent from large applicant pools. However, the underlying algorithm learns to favor male candidates over female ones.
The bias stems from the data used to train the machine learning model, which relies on resumes submitted to Amazon over a 10-year period. Because the tech industry historically skews male, the majority of these resumes belong to men, causing the AI to conclude that male resumes represent the ideal candidate. As a result, the system penalizes resumes containing the word "women's" and downgrades graduates of all-women's colleges.
Engineers attempt to correct this gender bias but ultimately fail to guarantee a fair outcome for all applicants. Executives lose hope in the project and officially disband the team early last year. Amazon confirms the tool never leaves the trial phase and states the company ultimately scraps it because it fails to deliver strong enough candidates.