Anthropic’s recent announcement that future Claude models will include invisible watermarks has raised questions about side effects beyond content provenance. A new analysis from Lasso Security examines how these watermarks—specifically Google DeepMind’s SynthID-Text technology—can unexpectedly alter AI agent behavior.
The EU AI Act requires providers of systems generating synthetic text to embed machine-readable watermarks that are detectable as artificially generated. However, according to the research, the watermarking process itself changes how models select tokens during generation, creating what researchers call “sampling drift.”
During generation, each token is selected sequentially based on probability distributions. The watermarking process biases these selections to embed its signal. While watermarking designers argue this is “non-distortionary”—meaning it preserves the original token distribution on average—the research shows that under a specific watermark key, individual generations can still differ substantially.
This matters particularly for AI agents, which rely on language model outputs to make decisions. In structured outputs like JSON function calls, predictable elements (braces, keys, function names) are less affected, but variable elements (queries, numbers, file paths, recipients) often shift. A lexical variation in ordinary text can become a functionally different tool argument that an agent executes.
The researchers tested watermarking’s effects using paired experiments: one with watermarking enabled, one without, using identical seeds and batch composition. On tool-calling tasks from the Berkeley Function Calling Leaderboard, watermarking reduced accuracy on six of seven tested models. More significantly, paired disagreement rates—instances where the same task produced different results—averaged 6.5% across 21 model-temperature combinations, substantially larger than net accuracy changes.
For example, on Phi-4 at temperature 1.0, 16.8% of tool-calling verdicts differed between watermarked and unwatermarked versions, despite only a 2.87-point net accuracy loss. This discrepancy occurs because improvements in some cases offset declines in others.
The error types also vary by model. On Llama-3.1-8B, incorrect arguments dominated accuracy losses, while on Phi-4 and Granite-3.2-8B, malformed output caused the largest drops. The research also indicates watermarking can weaken model refusals to harmful requests, particularly under prompt injection attacks.
According to the analysis, this behavioral shift has implications for AI safety and security, particularly when watermarking is deployed at the model level across multiple applications and agents using the same underlying model.
Key facts
- Anthropic announced future Claude models will embed invisible watermarks based on Google DeepMind’s SynthID-Text technology
- Watermarking changes how language models select tokens during generation, potentially affecting both model behavior and agent tool-calling accuracy
- Research shows 6.5% average paired disagreement rate across tested model-temperature combinations, meaning tool-calling verdicts differed between watermarked and unwatermarked versions
- The EU AI Act requires synthetic text generators to mark outputs as artificially generated in machine-readable format
- Watermarking effects are model- and key-dependent, and can be obscured by aggregate scores when changes occur in opposite directions