265
OpenAI has built a text watermarking method to detect chatgpt written content
(www.tomshardware.com)
this post was submitted on 11 Aug 2024
265 points (95.5% liked)
Technology
59123 readers
2290 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This has been known in the ML space forever. LLMs don't actually output words/tokens, but probabilities for a long list of tokens, and the sampler picks one (usually the mostl likely token). And if you arbitrarily weigh these probabilities (eg 50% of possible token outputs are more likely than the other 50%, as a random example), it creates a "signature" in any text thats easy to measure. The sampler randomizes it a tiny bit, but that averages out in long texts.
It's defeatable. I'm sure if you maken enough OpenAI queries, you can find the bias. I think a paper already tackled this. But this likely will stop the lazy absures, aka 99% of abusers, who should just use some other LLM if they really care.
Another open secret in LLM land is that OpenAI is actually falling behind open research efforts, hence its hilarious it took them this long to implement something so simple.
Ye, it'll stop the casual abuse for all of 5 minutes. There are already tools to obscure the use of AI and, as you say, it won't take much to update them.
It's not so trivial if OpenAI cycles the logit bias or makes it really convoluted.
And it's not like certain "words" or language patterns are more probable with this method, its different than what any kind of human or words based algorithm would detect, which is what I suspect most "anti AI detection" software does.
Its doable... but seems inconvenient for a small business to keep up with. Maybe.
Remember they're doing this so that they can detect it themselves. I'm far from an expert, so maybe I'm misunderstanding something but the way I understand it, they'd be defeating their own tool if they go down this route. If they cycle the logit biases, how can they themselves detect if a random piece of text is generated? Which set of biases do they test?
At the end of the day, you're talking about raw text. There's no option to sign it, or embed metadata or anything like that. You can't even guarantee that you're seeing the complete sample, or even a single sample! If there is a fingerprint, it'll be detectable to anyone, and it'll be easily removed.
They can cycle a some biases (dozens?) and test them all. Detokenization is super cheap to run, its not AI or anything.
I'm trying to think of a good analogy for how this would work, and I kinda came up with one. This would be kinda like an image encoder that biases itself towards coding RGB values (0-255) as even numbers. Subtly, say 30% odd 70% even.
That's totally imperceptile to humans. And even a "small" sample of the image would carry this bias if pasted into a larger image verbatim, since the sample size is so large (just as the sample size for a bunch of tokens in text is pretty big.
And I'm not saying its fullproof... but if thats indeed what they're doing, I think its a decent way to detect "lazy" OpenAI abusers who aren't working so hard to scramble and defeat it.