New Research Shows ChatGPT Search Is Vulnerable to Hidden Text Manipulation
A recent investigation reveals that OpenAI's new ChatGPT Search feature generates misleading summaries and malicious code when tricked by hidden text on websites. This marks the first known demonstration of such an attack on a live AI-powered search product.
New research from The Guardian reveals that ChatGPT Search, the AI-powered search engine launched this month, is vulnerable to manipulation. The tool is designed to speed up browsing by summarizing web pages, but investigators find that bad actors can easily trick it into producing completely false results.
The vulnerability relies on inserting hidden text into websites, a well-known risk for large language models. When encountering this hidden text, ChatGPT Search ignores negative product reviews to generate entirely positive summaries and even outputs malicious code. This appears to be the first time such an attack is successfully demonstrated on a live AI search product.
OpenAI does not comment on this specific incident but states that it uses various methods to block malicious websites and is continuously improving its defenses. Meanwhile, industry observers note that Google possesses more experience dealing with these exact types of search manipulation tactics.