570
This new data poisoning tool lets artists fight back against generative AI
(www.technologyreview.com)
this post was submitted on 23 Oct 2023
570 points (86.4% liked)
Technology
59123 readers
2310 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Invisible changes to pixels sound like pure BS to me. I'm sure others know more about it than i do but I thought pixels were very simple things.
"Invisible changes to pixels" means "a human can't tell the difference with a casual glance" - you can still embed a shit-ton of data in an image that doesn't look visually like it's been changed without careful inspection of the original and the new image.
If this data is added in certain patterns it will cause ML models trained against the image to draw incorrect conclusions. It's a technical hurdle that will slow a casual adversary, someone will post a model trained to remove this sometime soon and then we'll have a good old software arms race and waste a shit ton of greenhouse emissions adding and removing noise and training ever more advanced models to add and remove it.
You can already intentionally poison images so that image recognition draws incorrect conclusions fairly easily, this is the same idea but designed to cripple ML model training.
You're right, in that pixels are very simple things. However, you and I can't tell one pixel from another in an image, and at the scale of modern digital art (my girlfriend does hers at 300dpi), shifting a handful of pixels isn't going to make much of a visible difference to a person, but a LLM will notice them.
LLM is the wrong term. That's Large Language Model. These are generative image models / text-to-image models.
Truthfully though, while it will be there when the image is trained, it won't 'notice' it unless you distort it significantly (enough for humans to notice as well). Otherwise it won't make much of a difference because these models are often trained on a compressed and downsized version of the image (in what's called latent space)
An AI model will "notice them" but ignore them if trained on enough copies with them to learn that they're not significant.
Pixels are very simple things, literally 3-5 3 digit numbers.
But pixels mean little too a generative AI - it's all about relationship between pixels. All AI are high dimensional shapes right now... If you break up the shape strategically, it'll poison the image
Will this poison pill work? Probably, for at least a while...
have you ever seen those composite images made by combining a huge number of other, radically different images in such a way that each whole image acts like one "pixel" of the overall image? i bet AI models 'see' those images very differently than we do.
A pixel has a binary representation. All of the significant bits for the pixel may not not be needed to display the color of that pixel so there is often excess that can be used or modified. A person wouldn’t see it but an AI reading just the binary would.