Catch the LLM if you Can: Watermarking LLMs

Speaker 1 [00:09]

And so before starting, I want to remember you to recall the lightning talks. Go to the booth and apply. Also by the end of the session, sorry, by the end of the session.

Speaker 2 [00:32]

on the D.

Speaker 1 [00:34]

And then...

Speaker 2 [00:35]

trying to murder

Speaker 1 [00:38]

to do it in this way so the remote and everything is a little bit organized yeah I do my best so we welcome the speaker right now a with an applause

Speaker 2 [01:01]

you for the introduction and hi everyone and welcome to my talk. I'm super excited to be here at PyCon Germany. I want to thank all the sponsors, volunteers, speakers and the Python Software Foundation staffs for making this possible and also a delight. About me, my name is Shubhushree and I have a QR code to my LinkedIn so I don't spend a lot of time here introducing myself so you know who I am so here's what to expect from the talk we will learn about what is watermarking in the context of large language models and why it matters how they work will go through two popular text watermarking techniques and look into existing challenges and future research directions. Okay, so let's jump right into it then. Even without seeing it, the success of large language models is obviously, everyone knows that. Still, we take a quick look at the numbers, how ChatGPT gained 1 million users in only five days in 2022, and how Gen AI is expected to grow in the future with unexpected exponential growth. So it clearly looks like a major success story. But is it really a success? Because when you follow media, you probably have come across headlines like these, how scammers are increasingly using Gen AI tools to synthesize entirely new content. Moreover, when AI models are being trained on data that includes content generated by earlier versions of themselves, over time, this recursive process will cause the models to drift further away from the original data distribution, losing the ability to accurately represent the world as it really is. And today I want to shed some light on these and share techniques on how you can differentiate between AI and human-generated content. now when you think of a watermark you might think of a logo on an image or a timestamp embedded in a metadata but if the goal is to prove that a content is generated by llm then metadata doesn't really solve that problem because it can easily be taken out and if you look at the landscape of what's really out there you either have watermarks that are visible but discreet sort of tucked down at the corner of an image which is relatively easy to crop out so it does not serve the purpose of robustly watermarking a content and equally if you have a very highly visible watermark they can also be quite intrusive for visual appeal of the content so they don't provide a viable solution either. So researchers came up with a very clever way of hiding statistical patterns onto the generator output, which is a very simple idea and surprised many people that this is actually possible. So we should expect it to work because watermarking techniques exploit the fact that most LLM generation is already probabilistic. So under the surface, the neural network is pushing itself to pick from a special set of tokens. But we also need to keep in mind that the role of entropy is important in water marking. For instance, if someone asks an LLM the capital of Germany, then you can't water mark that. But when a prompt can lead to different generations, out of these exponentially many different generations, you can encode a signal. So here's only a possibility of watermarking when the LLM itself could have made multiple choices. Now watermarking text has been significantly challenging compared to other modalities like images and audios. There are quite a few reasons for that. First, all the texts are more sparse compared to other modalities. For example, LLM generated texts usually have a few thousand tokens, whereas a single image of 256 pixels has around 65K potential pixels for watermark insertion. So text is significantly more sparse and have fewer places for embedding watermark information. Also texts are more sensitive to smaller changes compared to images. Slight change in token replacement in text can distort its meaning and fluency, whereas such small changes in image are often imperceptible. Moreover, watermark texts are also more vulnerable to detection and removal attacks, because of the same reason that images or video have more redundancy in their data. So the watermark can be distributed across many pixels, but you cannot do that for text. Now in order to create a good watermark, several design decisions have to be made. So thinking about some of the use cases of LLMs, we can already think of some properties like the quality of the text has to be maintained. So the watermark shouldn't degrade the content of the text that the language model is creating. So the watermark shouldn't alter the predictive capability of the language model. Secondly, it should be hidden but also detectable by the creator. should be able to detect it with high accuracy to reduce false positives as this could lead to for example students being wrongly accused of cheating. Third it has to be robust to small modifications in the text. It shouldn't be easy to remove the watermark from the generated text. Not only should be robust against intentional attacks like paraphrasing or removing parts of the text but also accidental removal like text rendering from some formatting tool. Generally the more robust a watermark gets the more difficult it is for the attacker to detect and remove it. Finally the watermark should also be lightweight. There shouldn't be any significant latency and no need to retrain the models. Now in literature there are tons of watermarking methods but here I will to try to briefly summarize only two categories because most of the watermarking techniques were an extension of these two ideas. So the first idea is by a group at the University of Maryland by Christian Bauer and his colleagues. This paper was published in 2023. So to understand the technique, you don't need to know much about how LLMs work. The language model here is assumed to be something that takes in some previous text which is the user input and produces more text. So the way in which it produces more text is one by one it chooses more likely next word based on previous words and it does this repeatedly. So let's say we generated TUM is a grade and this goes into an LLM and you get some kind of a distribution over the next token. Let's say the possible outputs are university, research, place, German, and a part of the previous token will be the watermark context. Some hash of this previous context and the secret key determine how the vocabulary is plated into a green list and a red list. Then a bias is added to the green tokens. Here in this figure, some tokens ended up green and those logits are boosted. So the model is now more likely to generate a token out of the green set than the red. This doesn't modify the likelihood of the words in the vocabulary a lot. We just say that out of the most likely things according to the model make some of them more likely and some of them less likely. When enough of these choices are made following the slight biasing and breaking into green and red tokens then the text is generated with this system will actually contain a surprising number of word choices that come from the preferred set at every point of time in the text. So we can also create a nice statistical test based on this. So if the text is written by a real person or generated by some model which does not use this underlying watermarking technique then there will be a 50-50 chance that the next word that the model chooses is green and half the time the next word that is chosen is red. So there is nothing interesting going on there because without the logit post the token choice is determined solely by the model's learned distribution which has no reason to align with any particular key derived partition. But in the cases where an LLM with this specific KGW watermarking scheme has been used, then there's going to be this sort of surprising number of times where the word that appears is present in the green set that we can sort of compute and check. Taking in the previous words in the passage and compute a sort of grouping operation and checking is the word here green or is it marked as red. So that will allow us to develop a statistical grounded confidence estimate where we can say that it's very, very unlikely that the text which contains all of these preferred choices would have been just written by human or generated by some model that doesn't have this watermarking scheme. And then we can use a z-score and a large positive z-score will indicate the presence of a watermark. And the detector then rejects the null hypothesis and concludes that the text is watermarked when this Z-score exceeds the threshold and we can choose the threshold based on the desired false positive rate. So here it's like each token is a single Bernoulli trial. We just ask if it's a screen and with only 20 tokens there might be random fluctuations which can dominate. But with 200 or more tokens, the law of large numbers ensures that the observed green rate converges to a probability that would have been assigned by the LLM. So here we will see how with few lines of codes in Python we can implement the watermarking scheme. Also there's a link below in the presentation where you can find the code that we will discuss here. So first we are controlling two important hyperparameters, the gamma and delta. The gamma gives the fraction of tokens that we will call the green tokens. So if gamma is small, the watermark is very strong because we only allow a few green tokens, but the text can feel unnatural because the model has no limited choice. And if gamma is large, the model has many choices so the text flows naturally but direction becomes harder the next type of parameter is delta it tells us how much we boost these green tokens in the model's probability distribution so a large delta makes green tokens almost guaranteed which is useful for short sequences because you don't have enough tokens to accumulate evidence so you need a strong per token bias to make the watermark detectable but again the text can sound artificial so value of gamma equal to 0.5 and delta equal to 2 is considered to be an optimal choice through illustration we can also show how the z square distribution under the h0 and h1 hypothesis shift as the logit boost delta increases So with the logit post of delta equal to 0.5, the two distributions here nearly overlap, meaning the detector struggles to separate the watermark from the non-watermark text at this weak post level. For delta equal to 1, the watermark distribution, the one that is shown in blue, shifts noticeably rightward, creating a visible gap above the detection threshold, which is shown by the dotted line at z equal to 2.5. So, the true positive rate here will substantially be increased and for delta equal to 2 at this boost level the watermark distribution has shifted far enough that most probability mass lies above the detection threshold enabling reliable detection with high true positive rate and low false positive rate. The logit boost above this level is strong enough to measurably distort the output distribution. So this is a tradeoff that developers must consciously navigate. Now next to get the random number generator, the cRNG function takes the last token from the input sequence, which gives the context needed for watermarking and combines it with a fixed hash key to seed the random number generator. This prepares the RNG for generating reproducible green and red token lists. It ensures that the random process is deterministic, but also input dependent. For instance, after the word, say, apple, one set of green lists is created, and for the word banana, another completely different set of green lists should be created. And the hash key is chosen as a large prime number, which makes sure tokens give very different seeds, avoiding patterns in the random selection. This function here generates the green list, which is the set of tokens that the water marking process favors during generation. First it seeds the random number generator using the seed RNG function that we saw in the previous slide, then it calculates how many tokens should be green by multiplying the vocabulary size by the gamma parameter. A random permutation of all token indices is created using the seeded RNG and the first segment of this permutation is selected as the green list. Now next we apply the bias delta to the green tokens during generation using the bias green list logits function. The scores are the model's logits. The green list mask is a Boolean mask that tells us which tokens belong to the green list and this mask is created by comparing the vocabulary against the green list generated by the random number generator. So here we simply add a small bias to those green tokens, making them more likely to be selected by the model. So the key idea is that we are not changing the model drastically, we are just nudging it slightly towards certain tokens. So that over time the small nudges accumulate and insert a watermark in the generated text which can later be detected statistically. Now this function score sequence is where we check a sequence of tokens for the watermark. First we set a minimum prefix length, usually one token in the simplest scheme. We need this prefix because it seats the random generator that determines the green list. Starting after this prefix, we iterate through each token in the sequence. At each step, we compute the green list based on the tokens in the current prefix then check if the current token is part of this green list. If it is, we mark it as a green token and update the green token count. Otherwise, it's marked as red. By the end, we have both a total count of green tokens and a green token mask that shows exactly which tokens match and the green token mask is later used specifically for visualization and further analysis. Now, after we have iterated through the token sequence and counted how many tokens match the watermark, we summarize the results, and we compute the Z-score, which measures how much the observed number of green tokens deviates from what we would expect by chance. A high Z-score indicates that the watermark pattern is statistically significant. Next, we also compute the p-value, which tells us the probability of seeing this many green tokens if there were no watermark. So a smaller p-value, the more confident we are that the watermark is present. And the score sequence function then returns the score text, giving us all the information needed to detect and interpret the watermark in a text sequence. Now before we analyze the text for watermark, we first apply a set of normalizers. These are important because they protect the watermark detection from simple attacks or inconsistencies in the input text. For example, removing unusual or invisible characters like zero-bit spaces which could otherwise alter the tokenization without changing how the text looks. Or replacing visually similar characters from a different alphabet like a Cyrillic A that looks identical to a Latin A to ensure consistency. So also fixing irregular capitalization patterns such as random uppercase and lowercase letters, et cetera. So after normalization, the cleaned text is tokenized into input IDs using the model's tokenizer. We then remove the beginning of sequence token if it exists since it is a special marker used by the model and not part of the actual content. So these steps ensure that the watermark detection operates on a clean and consistent representation of the text. Next, the function calls the score sequence which computes the statistical matrices like the z-score and the p-value, based on how strongly the text follows the watermark pattern. These results are stored in an output dictionary. Finally, the function performs a hypothesis test by comparing the z-score to a threshold, and if it exceeds the threshold, the text is classified as watermarked and a confidence score is computed. The function then returns all results, including the prediction and associated matrices, and this is how we can implement the KGW watermarking technique. Now, the next technique is this idea developed by Aronson and Kirchner, where the language model's probability distribution is modified rather than forcing specific token choices. So it relies on selecting tokens that maximize hash function. We begin with a prompt, which is partially generated sequence. The language model processes the context and produces a logic vector for the next token. Now using a secret key and the current context, a pseudo-random function generates a watermark key. So the pseudo-random function takes the secret key and context as input. So the pseudo-random function takes the secret key and the context at input and deterministically generates vector that looks like independent and identically distributed uniform samples. The decoder then combines the model probability with the watermark key. This effectively injects controlled randomness into the sampling process while keeping the tokens consistent with the language model distribution. The modification is small enough that the text quality is preserved, but large enough to leave a detectable statistical trace. In expectation, the generated tokens still follow the original distribution, however conditioned on the watermark key, they exhibit a structured bias that can later be detected. Now at the detector, by using the same secret key and the observed text context, the detector reconstructs the same watermark keys. For each token, we compute a score that measures how consistent the token is with the watermark key. And then we combine all bar token scores into a single global statistics which is simply the sum of the bar token scores and if the aggregated statistic exceeds the threshold then the detector concludes that the text contains the watermark. And we can set the threshold depending on what the probability of a false positive or false negative we are willing to tolerate. So if we now want to put the watermarking techniques on this diagram, we see that the KGW method has decent quality. Not great because we will suffer when delta and gamma parameters are not chosen correctly. The detectability and robustness are both very high, but since we are using a simple watermark and people can learn the green list as you observe more and more data and adaptively query the language model. So the security against learning is relatively on the lower end. For the EXP watermarking technique, the detectability is high and the quality is also much higher than the green-red watermark because we sample in a way consistent with the original distribution. It's also more secured because we have more secret bits of information. It has larger number of bits compared to the KGW method, which encodes a single bit of information. And as we increase the length of the prefix, the security also gets better, but the robustness will suffer because the KGW technique is, there is no way that you can do a unigram watermark for the EXP without making everything that you generated just completely deterministic. Now next there is a toolkit if you want to try other watermarking techniques you can check out the mark LLM GitHub repository. It is a unified framework for experimenting with different watermarking techniques. It supports multiple algorithm families including the KGW and the EXP method that we talked about and it also provides a user-friendly interface where you can load an algorithm, a watermark to the text, detect it and even visualise the mechanism showing which parts of the text are affected and beyond visualisation they also have automated evaluation tools for robustness like word deletion, synonym substitution and paraphrasing and for text quality including some GPT based assessments and including calculation of blue scores. So it also supports pipelines for watermark detection and text quality analysis, making experimentation quite smooth. So now what about deployment of these models? So without going into too much details, I will go through some of the challenges that watermarking techniques are currently facing. The technical limitations include trade-offs between robustness and quality. Strong watermarks designed to resist watermark removal attempts can noticeably degrade the quality of the underlying text, impacting readability and user experience, whereas subtle watermarks preserve the high text quality but are more susceptible to watermark bypassing. Attackers may employ various strategies such as fine-tuning the model on a new dataset, tuning parameters to reduce model size or using knowledge distillation to transfer knowledge to a smaller watermark-free model. These attacks can potentially remove or alter the embedded watermark. The second problem is that an absence of standardized benchmarks and evaluation protocols, which significantly hinders the ability to perform fair comparisons. There is still no unified benchmark widely adopted by the research community, where the watermarking efforts differ in task coverage, evaluation criteria, and attack assumptions, limiting their ability to be generalized and compared with the published results. And then comes the ethical considerations, while watermarks can protect the intellectual property, but they can also raise privacy implications. Secretly embedding, identifying information in text could be used to track readers or reveal sensitive data without their knowledge or consent. So is it unfair to some people who are, for instance, learning a language and using AI to write emails in some other language? So do they really want it to be watermarked? So there is no watermarking tool that works for those sympathetic kind of cases. And also, who gets access to the watermark tools and can all the AI companies coordinate to use one particular watermark? So, to summarize, despite these limitations, watermarking has opened several important research directions and has changed how the field thinks about AI watermarking. I think it has made some progress in developing watermarking models without degrading the quality and attacks. Certainly possible, but also better watermarking schemes can be, we can come up with better watermarking scheme. So, till now there is no clear winner. So, yeah, thank you and I'm looking forward to hearing your ideas and also you can ask questions.

Speaker 1 [27:38]

So here, sorry, for our first question, does the first watermarking method not only work for the actual model provider disease, you could, as a third party,

Speaker 2 [27:52]

You could as a third party.

Speaker 1 [27:54]

without access to the model not verify this.

Speaker 2 [28:09]

yes as a third party you can also like since it is open source so yeah you can access them

Speaker 1 [28:21]

Thank you. One second, please. Does watermarking help to identify text generated by a specific LLM, or is the goal to distinguish between AI and human-generated tests?

Speaker 2 [28:51]

The goal is to distinguish between AI and human-generated text, so it does not apply to a specific LLM, but currently it is more of a research-based, so if you already have an idea of, for example, in the KTW, you already know how it is working, so in this case it is more dependent on the lm but the main goal is to differentiate between ai and

Speaker 1 [29:31]

you I think this one is related to the circumvent what a marking can't I simply ask another model to rephrase the text for rephrase all I only need a small maybe open model of which I know of which I know it doesn't use

Speaker 2 [29:57]

Yes, so that is what I...