Technology

61227 readers

4102 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

[email protected]

724

AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt (arstechnica.com)

submitted 2 days ago by [email protected] to c/[email protected]

84 comments fedilink hide all child comments

Building on an anti-spam cybersecurity tactic known as tarpitting, he created Nepenthes, malicious software named after a carnivorous plant that will "eat just about anything that finds its way inside."

Aaron clearly warns users that Nepenthes is aggressive malware. It's not to be deployed by site owners uncomfortable with trapping AI crawlers and sending them down an "infinite maze" of static files with no exit links, where they "get stuck" and "thrash around" for months, he tells users. Once trapped, the crawlers can be fed gibberish data, aka Markov babble, which is designed to poison AI models. That's likely an appealing bonus feature for any site owners who, like Aaron, are fed up with paying for AI scraping and just want to watch AI burn.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 23 points 3 hours ago (1 children)

There are different kinds of AI scraper defenses.

This one is an active strategy. No shit people know that this costs them resources. The point is that they want to punish the owners of bad-behaved scrapers.

There is also another kind which just blocks anything that tries to follow an invisible link that goes to a resource forbidden by robots.txt

[+] [email protected] -11 points 2 hours ago (1 children)

One or two people using this isn't going to punish anything, or make enough of a difference to poison the AI. That's the same phrase all these anti-AI projects for sites and images use, and they forget that, like a vaccine. you have to have the majority of sites using your method in order for it to be effective. And the majority of sysadmins are not going to install what's basically ICE from Cyberpunk on a production server.

Once again, it's lofty claims from the anti-AI crowd, and once again it's much ado about nothing. But I'm sure that won't stop people from believing that they're making a difference by costing themselves money out of spite. 😂

[–] [email protected] 4 points 1 hour ago* (last edited 1 hour ago) (1 children)

The only AI company that responded to Ars' request to comment was OpenAI, whose spokesperson confirmed that OpenAI is already working on a way to fight tarpitting.

Ah yes. It's extremely common for one of the top companies in an industry to spitefully expend resources fighting the irrelevant efforts of...

One or two people

Please, continue to grace us with you unbiased wisdom. Clearly you've read the article and aren't just trying to simp for AI or start flame wars like a petulant child.

[–] [email protected] 0 points 1 hour ago

Well, luckily for them, it's a pretty simple fix. Congrats on being a part of making them jot down a note to prevent tarpitting when they get around to it. You've saved the internet!

And stop pretending like you're unbiased either. We both have our preconceived notions, and you're not more likely to be open to change yours than I am. In fact, given the hysterical hyperventilating anti-AI "activists" get to, we both know you're not ever going to change your mind on AI, and as such you'll glom onto any small action you think is gonna stick it to the man, no matter whether that action is going to have any practical effect on the push for AI or not.