AI Text Watermarking Weakens LLM Safety Guardrails, Researchers Find
AI platforms are rolling out watermarking schemes in response to a new European Union law, with Anthropic announcing that future Claude models will use SynthID-Text, an open-source approach created by Google. The technique uses a secret key that subtly alters how a model selects the next word in a sentence, allowing anyone with the key to verify that content was AI-generated. While the watermark is designed to be imperceptible to readers, researchers are now finding it has broader side effects than intended.
Research from Lasso Security shows that SynthID-Text changes not only word selection but also the tools a model invokes and how faithfully it follows safety guardrails. Under adversarial prompts, where attackers try to trick a model into harmful actions like revealing passwords, models that would normally refuse instructions sometimes comply once watermarking is enabled. "Watermarking is made to not be perceptible to a reader, but we know that when we are changing anything about what the model is generating, it is going to cause some tradeoffs," Lasso AI security researcher Andrea Siposova tells Ars.
The findings highlight a "provenance tax" that developers must account for: any change to a model's generation process carries behavioral consequences. The results underscore the need for AI developers to thoroughly test how their LLMs and agents behave with watermarking in place, especially in adversarial conditions and when models are powering tool-using agents.