New ImageNet Roulette Tool Exposes Deep Bias in AI Image Labeling
A new online tool called ImageNet Roulette allows users to upload selfies to see how artificial intelligence labels them, revealing deeply offensive and biased results. The project aims to spark a conversation about the problematic training data that powers modern image recognition systems.
A new online tool called ImageNet Roulette highlights the inherent bias in artificial intelligence by allowing users to upload a selfie and see how AI labels them. While some results are accurate or humorous, such as labeling U.K. Prime Minister Boris Johnson as a "demagogue," other outcomes are blatantly offensive and racist. People with darker skin frequently receive harmful labels like "wrongdoer," "offender," or "convict," exposing the prejudices embedded within the system.
The project is the work of AI Now head Kate Crawford and researcher Trevor Paglen, created as part of the "Training Humans" art exhibition at the Fondazione Prada Osservertario museum in Milan. The tool relies on the massive ImageNet database, an archive of 14 million labeled images that uses WordNet, a word classification system developed in the 1980s. Crawford and Paglen designed the experiment to make the invisible process of AI classification visible to the public.
The creators emphasize that ImageNet Roulette does not strive for perfection, but instead serves as a deliberate critique of problematic training data. This tool is hardly the first instance of AI bias making headlines, as similar controversies have recently plagued other tech giants like Google. By shedding light on these flawed systems, the project hopes to spark a much-needed discussion about the ethical implications of using biased algorithms in everyday technology.