Amazon Scraps AI Recruiting Tool After It Learns to Discriminate Against Women
Amazon abandons an experimental AI recruiting engine after discovering it automatically penalizes female candidates. The system learns this sexist behavior by training on a decade of male-dominated tech industry resumes.
Amazon abandons a secret artificial intelligence recruiting project after the system teaches itself to discriminate against female candidates. The experimental AI relies on machine learning to review resumes and rate applicants on a scale of one to five stars, but it quickly develops a strong preference for men.
The bias stems directly from the training data, which consists of a decade's worth of resumes submitted to the male-dominated tech industry. As a result, the AI actively downgrades resumes containing the word "women's" and penalizes graduates of all-women's colleges, while rewarding verbs like "executed" and "captured" that appear more frequently in male engineers' resumes.
Engineers attempt to fix the algorithmic bias by editing the system, but they ultimately find it impossible to prevent the AI from discovering new ways to penalize female applicants. Although Amazon insists the tool never actually evaluates real job seekers, the company officially kills the project in early 2017 as other tech giants like Facebook also ramp up efforts to detect similar biases in their own hiring algorithms.