The qualifier is there because it changes the outputs, so it’s necessarily true that some outputs will be worse.

But it’s just as likely to make an output better.

Take the example from the article. He complains that watermarking might sometimes, for example, choose to say “bananas” over “pineapples” because only the former is on the green list, potentially making an output less precise. But 1. It could do that regardless of watermarking since the model is probabilistic, and 2. The more accurate word choice of “pineapples” is equally likely to be on the green list instead, further increasing its likelihood!

Overall, the article is pretty silly because he’s complaining about the possibility of Claude not always choosing the most “optimal” token, even though LLMs are probabilistic so that will happen anyways.

> But it’s just as likely to make an output better.

No, for any particular output token the model's true logits are definitionally the 'best' that the model can achieve.

This is inherently probabilistic. The model's top-1 guess is not guaranteed to be optimal, but it should be so a proportionate fraction of the time. Same with the top-2, top-3, etc.

Watermarking necessarily alters the output distribution away from the model-set distribution, and that alteration is inherently 'worse' in expectation.

You can liken this to a weather forecast. If there's a 25% chance of rain, the forecast should say so (or a 'sampled' deterministic forecast should predict rain 25% of the time). If the forecast is 'watermarked' and predicts rain 27% of the time under identical circumstances, it's a worse forecast.

That being said, this is a case of hiding a message in a noisy channel. Watermarking only needs to communicate one bit ('yes watermark'), so the effects can be arbitrarily small provided one is willing to tolerate an increase to the text size needed for reliable detection.

I think you can also just use the random number generator (seeded with a secret key) as the watermark. Then the probability distribution is exactly the same.

This assumes "writing quality" were somehow an individual property of each word, and the writing quality of the entire text would just be that property for each word summed together.

Which is obviously not how it works.

> He complains that watermarking might sometimes, for example, choose to say “bananas” over “pineapples” because only the former is on the green list, potentially making an output less precise.

If I replace "pineapples" with "bananas" in any kind of meaningful text, I've not made the text "less precise", I made it plain wrong. An incorrect statement. And if the words next to it are still correct or even somehow replaced with even more correct words, the text in its entirety will still be wrong.

> It could do that regardless of watermarking since the model is probabilistic

No, because the probability distribution of the model is generated by its training data and contains semantic information. So the model choosing a completely incorrect word is unlikely. However the probability distribution of the red/green lists is not guided by semantic information.