wingiee

joined 4 weeks ago
sorted by: new top controversial old
Everyone knows all the apps on your phone in c/[email protected]
Cloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives. in c/[email protected]
[–] [email protected] 20 points 4 weeks ago (2 children)

I assume a crawler which adheres to robots.txt