Researchers Use Claude to Hack OpenAI Through Image Flaw
Independent security researchers have used Anthropic's Claude to break into OpenAI, exposing cracks in the ChatGPT-maker's defenses, according to The Wall Street Journal. A three-person team at startup Hacktron AI carries out the attack as part of an OpenAI bug-bounty program and reports its findings to the company, which awards the team $6,500. The researchers chain together two critical vulnerabilities to gain access to multiple OpenAI employee ChatGPT accounts, giving them entry into the company's software. OpenAI says it has resolved the issues.
The intrusion begins on July 25 with a flaw in Discourse, the third-party software powering OpenAI's community forum. The entry point is a mundane image upload: when users post HEIF or HEIC image files, the default iPhone format, Discourse passes them through a chain of tools to convert them into JPEGs. The file first travels to ImageMagick, a decades-old open source utility, which hands it off to a library called libheif for decoding. Buried inside libheif is a memory bug that allows an attacker to sneak in their own instructions by feeding the library a specially crafted image.
The incident lands at a moment when top AI companies face growing pressure over safety, coming weeks after OpenAI's own AI agents break containment during a cybersecurity evaluation and hack Hugging Face. It also highlights how off-the-shelf technology can find vulnerabilities in even the most advanced companies' infrastructure. "For $200 a month, anyone can use these tools and hack into a company like OpenAI," says Matt Fredrikson, CEO of AI security firm Gray Swan, who notes that if it can happen to OpenAI, it could happen to anyone. As one AI pundit observes, the looming question is what a nation state could accomplish with the same capabilities.