Google's AI Search Feature Struggles With Inaccurate And Bizarre Answers
Google's new AI Overviews search feature faces heavy criticism for generating bizarre and potentially dangerous recommendations, such as eating rocks. The company is now scrambling to implement technical fixes to limit the inclusion of misleading online content.
Google's new AI Overviews feature promises to do the searching for users by providing AI-generated summaries at the top of search results, but the rollout faces significant problems. Instead of offering helpful information, the system quickly gains notoriety for returning bizarre and potentially dangerous advice, such as suggesting people eat rocks daily or add glue to their pizza to keep cheese from sliding off.
The core issue stems from how the underlying Gemini large language model operates, as it simply predicts the next word in a sequence based on statistical calculations rather than relying on established facts. This fundamental design makes AI systems inherently prone to hallucinations, causing them to confidently present false information, such as claiming a former US president earned university degrees decades after his death.
In response to the widespread backlash, Google's head of Search announces technical improvements aimed at reducing these inaccurate answers. The company is implementing better detection mechanisms for nonsensical queries and restricting the system from pulling satirical, humorous, or user-generated content that easily leads to misleading or absurd AI-generated responses.