github/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-12-29 12:18:33 +01:00
Code Issues Projects Releases Packages Wiki Activity

Update from Dark Visitors

Browse source
This commit is contained in:
ai.robots.txt 2025-06-29 01:07:47 +00:00
commit 2bfcee794d
6 changed files with 21 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
haproxy-block-ai-bots.txt
View file

@ -6,6 +6,7 @@ Andibot
anthropic-ai
Applebot
Applebot-Extended
Awario
bedrockbot
Brightbot 1.0
Bytespider
@ -19,8 +20,11 @@ cohere-ai
cohere-training-data-crawler
Cotoyogi
Crawlspace
Datenbank Crawler
Devin
Diffbot
DuckAssistBot
Echobot Bot
EchoboxBot
FacebookBot
facebookexternalhit
@ -43,6 +47,7 @@ meta-externalagent
Meta-ExternalAgent
meta-externalfetcher
Meta-ExternalFetcher
MistralAI-User
MistralAI-User/1.0
MyCentralAIScraperBot
NovaAct
@ -73,6 +78,7 @@ Sidetrade indexer bot
TikTokSpider
Timpibot
VelenPublicWebCrawler
WARDBot
Webzio-Extended
wpbot
YandexAdditional