AI Models Show Higher Inaccuracy Rate on Election Questions in Spanish
A new study reveals that leading AI models provide incorrect answers to election-related questions more frequently when asked in Spanish than in English, raising concerns about language-based bias.
A new study from the AI Democracy Projects reveals that leading AI models struggle significantly more to answer election-related questions accurately in Spanish compared to English. Researchers tested five major generative models, including tools from Anthropic, Google, OpenAI, Meta, and Mistral, using 25 prompts designed to mimic questions an Arizona voter might ask ahead of the U.S. presidential election.
The results show a clear disparity in performance based on language, with 52% of the Spanish-language responses containing factually wrong information. In contrast, only 43% of the English-language responses to the exact same questions provide incorrect details. Questions range from simple civics topics like the Electoral College to specific voting procedures such as the definition of a federal-only voter.
This gap in accuracy highlights a surprising form of bias within AI systems and underscores the potential harm such errors cause for Spanish-speaking voters. As generative AI becomes a more common tool for finding voting information, these language-based inaccuracies threaten to mislead a significant portion of the electorate and disrupt the democratic process.