DeepMind debuts watermarks for AI-generated text

These watermarks are not robust to paraphrasing attacks: AUC ROC falls from 0.95 to 0.55 (barely better than guessing) for a 100 token passage.<p>The existing impossibility results imply that these attacks are essentially unavoidable (<a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.04378" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.04378</a>) and not very costly, so this line of inquiry into LLM watermarking seems like a dead end.

This article goes into it a little bit, but an interview with Scott Aaronson goes into some detail about how watermarking works[0].<p>He&#x27;s a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together slightly more than random and you can mathematically derive with some level of certainty whether an output or even a section of an output was generated by the LLM. It&#x27;s really clever and apparently he has a working prototype in development.<p>Some work arounds he hasn&#x27;t figured out yet is asking for an output in language X and then translating it into language Y. But those may still be eventually figured out.<p>I think watermarking would be a big step forward to practical AI safety and ideally this method would be adopted by all major LLMs.<p>That part starts around 1 hour 25 min in.<p>&gt; Scott Aaronson: Exactly. In fact, we have a pseudorandom function that maps the N-gram to, let’s say, a real number from zero to one. Let’s say we call that real number ri for each possible choice i of the next token. And then let’s say that GPT has told us that the ith token should be chosen with probability pi.<p><a href="https:&#x2F;&#x2F;axrp.net&#x2F;episode&#x2F;2023&#x2F;04&#x2F;11&#x2F;episode-20-reform-ai-alignment-scott-aaronson.html" rel="nofollow">https:&#x2F;&#x2F;axrp.net&#x2F;episode&#x2F;2023&#x2F;04&#x2F;11&#x2F;episode-20-reform-ai-ali...</a>

&quot;An LLM generates text one token at a time. These tokens can represent a single character, word or part of a phrase. To create a sequence of coherent text, the model predicts the next most likely token to generate. These predictions are based on the preceding words and the probability scores assigned to each potential token.<p>For example, with the phrase “My favorite tropical fruits are __.” The LLM might start completing the sentence with the tokens “mango,” “lychee,” “papaya,” or “durian,” and each token is given a probability score. When there’s a range of different tokens to choose from, SynthID can adjust the probability score of each predicted token, in cases where it won’t compromise the quality, accuracy and creativity of the output.<p>This process is repeated throughout the generated text, so a single sentence might contain ten or more adjusted probability scores, and a page could contain hundreds. The final pattern of scores for both the model’s word choices combined with the adjusted probability scores are considered the watermark. This technique can be used for as few as three sentences. And as the text increases in length, SynthID’s robustness and accuracy increases.&quot;<p>Better link: <a href="https:&#x2F;&#x2F;deepmind.google&#x2F;technologies&#x2F;synthid&#x2F;" rel="nofollow">https:&#x2F;&#x2F;deepmind.google&#x2F;technologies&#x2F;synthid&#x2F;</a>

Some comments here point at impossibility results, but after screening hundreds of job applications at work, it&#x27;s not hard to pick out the LLM writing, even without watermark. My internal LLM detector is now so sensitive that I can tell when my confirmed-human colleagues used an LLM to rephrase something when it&#x27;s longer than just one sentence. The writing style is just so different.<p>Maybe if you prompt it right, it can do a better job of masking itself, but people don&#x27;t seem to do that.

Some of the watermarking is really obvious. If you write song lyrics in ChatGPT, watch for phrases like &quot;come what may&quot; and &quot;I stand tall.&quot;<p>It&#x27;s not just that they are (somewhat) unusual phrases, it&#x27;s that ChatGPT comes up with those phrases so very often.<p>It&#x27;s quite like how earlier versions always had a &quot;However&quot; in between explanations.

The academic paper: <a href="https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s41586-024-08025-4" rel="nofollow">https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s41586-024-08025-4</a><p>They use the last N prefix tokens, hash them (with a keyed hash), and use the random value to sample the next token by doing an 8-wise tournament, by assigning random bits to each of the top 8 preferred tokens, making pairwise comparisons, and keeping the token with a larger bit. (Yes, it seems complicated, but apparently it increases the watermarking accuracy compared to a straightforward nucleus9 sampling.)<p>The negative of this approach is that you need to rerun the LLM, so you must keep all versions of all LLMs that you trained, forever.

This is information-theoretically guaranteed to make LLM output worse.<p>My reasoning is simple: the only way to watermark text is to inject some relatively low-entropy signal into it, which can be detected later. This has to a) work for &quot;all&quot; output for some values of all, and b) have a low false positive rate on the detection side. The amount of signal involved cannot be subtle, for this reason.<p>That signal has a subtractive effect on the predictive-output signal. The entropy of the output is fixed by the entropy of natural language, so this is a zero-sum game: the watermark signal will remove fidelity from the predictive output.<p>This is impossible to avoid or fix.

Google is branding this in a positive light but this is just AI text DRM.

I think we just need to give up on this. What’s the harm? It’s not like some ground truth is fabricated.<p>I’m far, far more concerned about photo, video, and audio verification. We need a camera that can guarantee a recording is real.

&gt; the team tested it on 20 million prompts given to Gemini. Half of those prompts were routed to the SynthID-Text system and got a watermarked response, while the other half got the standard Gemini response. Judging by the “thumbs up” and “thumbs down” feedback from users, the watermarked responses were just as satisfactory to users as the standard ones.<p>Three comments here:<p>1. I wonder how many of the 20M prompts got a thumbs up or down. I don&#x27;t think people click that a lot. Unless the UI enforces it. I haven&#x27;t used Gemini, so I might be unaware.<p>2. Judging a single response might be not enough to tell if watermarking is acceptable or not. For instance, imagine the watermarking is adding &quot;However,&quot; to the start of each paragraph. In a single GPT interaction you might not notice it. Once you get 3 or 4 responses it might stand out.<p>3. Since when Google is happy with measuring by self declared satisfaction? Aren&#x27;t they the kings of A&#x2F;B testing and high volume analysis of usage behavior?

Correct me if I’m wrong, but wouldn’t it simply drive people to use LLMs that are not watermarking their content?

This strikes me as potentially a bad thing for regular people. For example, corporations call still use AI filtering to force job seekers to jump through hoops but job seekers won&#x27;t be able to use AI to generate the cover letters and resumes that those hoops demand.

To archive the watermark they store every output which they create and let partners check against it. That’s how I understand the article.<p>Then they also store everything which the partners upload to check if it’s created by them.<p>If other AI players also would store everything they create and make it available in a similar way there could be indeed some working watermark.<p>If one would use a private run AI to change the public run AI generated content to alter it there still would be a percentage similarity recognisable to hint that it might come from one of the public AIs.<p>Timestamps would become quite relevant since much content would start to repeat itself at some point and the answers generated might be similar.

By design, a watermark would make it easy to create a discriminator that distinguishes between LLM content and human content. In that case, just make a discriminator yourself and use regex to find and remove any of the watermarks.

I think people are already doing that. I frequently hear people watermarking their speeches with phrases like &quot;are we aligned on this?&quot;, or &quot;let&#x27;s circle back&quot; and similar.

&gt; Such modifications introduce a statistical signature into the generated text,<p>Great so now people have to be worried about being too statistically similar to an arbitrary &quot;watermark&quot;.

I really want to be able to try Gemini without the AI watermark. IIRC they&#x27;ve used SynthID from the start and it makes me wonder if it&#x27;s the source of all of Gemini&#x27;s issues.<p>Obviously Google claims that it doesn&#x27;t cause any issues but I&#x27;d think that OpenAI and other competitors would have something similar to SynthID if it didn&#x27;t impact performance.

I want AI to use just the right word when it’s writing for me. If it’s going to nerf itself to not choose the perfect word so it can be watermarked, then why would I use that product? I’ll go somewhere else. And if it does use just the right word, then how is that different from a great human writer?

Google are obviously pushing this as a way to root out AI blog spam.<p>If only they can get other providers to use it because of &#x27;safety&#x27; or something they won&#x27;t have to change their indexer much. Otherwise page rank is dead due to the ease of creating content farms.

Do LLM&#x27;s always pick the most probable next word? I would have thought this would lead to having the same output for every input? How does this deal with the randomness that you get from prompting the same thing over and over?

&gt; It has also open-sourced the tool and made it available to developers and businesses, allowing them to use the tool to determine whether text outputs have come from their own large language models (LLMs), the AI systems that power chatbots. However, only Google and those developers currently have access to the detector that checks for the watermark.<p>These two sentences next to each other don&#x27;t make much sense. Or are misleading.<p>Yeah. I know. Only the client is open source and it calls home.

To what degree will AI-generated text and watermarking influence how human language evolves... I bet &quot;delve&quot; will become more frequent in the spoken language :)

OT: The publication (Spectrum by IEEE) has some really good content.<p>It&#x27;s starting to become a common destination for when I want to read about interesting things.

How is this supposed to work? By inserting special unicode characters?<p>How can you watermark text?

&quot;I hope this message finds you well.&quot; --- busted!

Who is going to pay for watermarked output?