Amazon Abandons AI Hiring Tool After It Learns to Discriminate Against Women
Amazon scraps an experimental artificial intelligence recruiting engine after discovering it automatically penalizes female job candidates. The system learns this bias by studying the tech industry's historically male-dominated resume pool.
Amazon abandons an experimental artificial intelligence hiring tool after discovering that the system discriminates against female job candidates. Engineers in Scotland develop the program in 2014 with the goal of mechanizing the recruitment process by automatically identifying the top talent from a pool of resumes. However, the project ultimately fails because the AI learns to favor men for technical roles.
The discrimination stems directly from the data used to train the machine learning algorithms. Because the tech industry suffers from a well-known gender imbalance, the historical resumes fed into the system belong predominantly to men. As a result, the AI teaches itself that male candidates are preferable and actively penalizes resumes that include the word "women's," such as "women's chess club captain."
The flawed program also downgrades graduates of two all-women's colleges and dismisses basic tech skills that appear on a wide variety of applications. Although recruiters look at the AI's recommendations while searching for talent, they never rely solely on its rankings before the company officially scraps the tool in 2017. This situation serves as a prominent cautionary tale about the risks of allowing computers to make critical human decisions.