Understanding the Impact of LLM Watermarking on AI Agent Behavior

Anthropic's future Claude models will embed an invisible watermark, based on Google DeepMind’s SynthID-Text, in their output. This deployment has regulatory relevance, aligning with Article 50(2) of the EU AI Act, which requires AI systems generating synthetic text to mark outputs as artificially generated. Research measures the impact using "churn," the paired disagreement rate between watermarked and unwatermarked runs. For example, at T=1.0, phi-4 showed 16.8% churn with a 2.87-point net accuracy loss, while Llama-3.1-8B had 9.9% churn and 0.87-point loss. Watermarking can also affect refusals, and prompt injection is identified as an input-side vulnerability.