github/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-12-29 12:18:33 +01:00
Code Issues Projects Releases Packages Wiki Activity
551 commits 6 branches 46 tags 797 KiB
Commit graph

551 commits

This branch
This branch
All branches
Author SHA1 Message Date
ai.robots.txt
e87eb706e3 Merge pull request #188 from ai-robots-txt/cdransf/buddybot 
chore: adds BuddyBot
 2025-11-04 05:16:28 +00:00
Glyn Normington
3d41350256
 		Merge pull request #188 from ai-robots-txt/cdransf/buddybot 
chore: adds BuddyBot
 2025-11-04 05:16:15 +00:00
Cory Dransfeldt
808451055c
 		chore: adds BuddyBot 2025-11-03 13:39:07 -08:00
ai.robots.txt
5cad0ee389 Merge pull request #187 from ai-robots-txt/cdransf/add-Linguee-Bot 
chore: adds Linguee Bot
 2025-10-24 18:05:01 +00:00
Glyn Normington
d55c9980cd
 		Merge pull request #187 from ai-robots-txt/cdransf/add-Linguee-Bot 
chore: adds Linguee Bot
 2025-10-24 19:04:42 +01:00
Cory Dransfeldt
192b0a2eef
 		chore: adds Linguee Bot 2025-10-24 10:35:04 -07:00
dark-visitors
97e19445ce Update from Dark Visitors 2025-10-24 00:53:30 +00:00
ai.robots.txt
0bc2361be8 Merge pull request #186 from ai-robots-txt/third-party 
Use third party in text, rather than first
 2025-10-21 15:23:46 +00:00
Cory Dransfeldt
511d8c955d
 		Merge pull request #186 from ai-robots-txt/third-party 
Use third party in text, rather than first
 2025-10-21 08:23:28 -07:00
Glyn Normington
b89a9eae6a Use third party in text, rather than first 2025-10-21 01:11:32 +01:00
ai.robots.txt
646ab08e15 Merge pull request #185 from ai-robots-txt/cdransf/add-IbouBot 
chore: adds IbouBot
 2025-10-21 00:09:24 +00:00
Glyn Normington
5c3da1c1af
 		Merge pull request #185 from ai-robots-txt/cdransf/add-IbouBot 
chore: adds IbouBot
 2025-10-21 01:09:06 +01:00
Glyn Normington
d22b8dfd7b
 		Description is third party, not first 2025-10-21 01:08:42 +01:00
Cory Dransfeldt
19c1d346c3
 		chore: adds IbouBot 2025-10-20 09:28:31 -07:00
ai.robots.txt
2fa0e9119c Merge pull request #183 from ai-robots-txt/cdransf/bot-additions-cloudflare v1.42 
chore: add amazon-kendra-, Anomura, Cloudflare-AutoRAG and Bravebot
 2025-10-20 14:34:43 +00:00
Glyn Normington
0874a92503
 		Merge pull request #183 from ai-robots-txt/cdransf/bot-additions-cloudflare 
chore: add amazon-kendra-, Anomura, Cloudflare-AutoRAG and Bravebot
 2025-10-20 15:34:30 +01:00
Cory Dransfeldt
28d2d09633
 		chore: add amazon-kendra-, Anomura, Cloudflare-AutoRAG and Bravebot 2025-10-19 15:57:28 -07:00
ai.robots.txt
260f5029fe Update from Dark Visitors 2025-10-16 00:56:33 +00:00
dark-visitors
91bf905fa9 Update from Dark Visitors 2025-10-15 00:56:30 +00:00
dark-visitors
56d03d46fb Update from Dark Visitors v1.41 2025-09-26 00:54:30 +00:00
ai.robots.txt
38d60b928c Merge pull request #179 from ai-robots-txt/deepseekbot 
chore(robots.json): add DeepSeekBot
 2025-09-25 10:17:38 +00:00
Glyn Normington
e2266bbc1d
 		Merge pull request #179 from ai-robots-txt/deepseekbot 
chore(robots.json): add DeepSeekBot
 2025-09-25 11:17:23 +01:00
Cory Dransfeldt
bf347bdf91
 		chore(robots.json): add DeepSeekBot 2025-09-24 09:42:01 -07:00
nisbet-hubbard
c6e7d69dd5
 		Update README.md 2025-09-24 09:39:21 -07:00
ai.robots.txt
8906f6b447 Update from Dark Visitors 2025-09-12 00:53:06 +00:00
dark-visitors
2fd93029ca Update from Dark Visitors 2025-09-11 00:55:06 +00:00
ai.robots.txt
b6338ddc73 Merge pull request #176 from emersion/TerraCotta 
Add TerraCotta
 2025-09-10 14:11:16 +00:00
Cory Dransfeldt
7ffbf33baf
 		Merge pull request #176 from emersion/TerraCotta 
Add TerraCotta
 2025-09-10 07:11:04 -07:00
Simon Ser
50870ba911 Add TerraCotta 
Example server log:

    X.X.X.X - - [10/Sep/2025:00:06:17 +0000] "GET /archives/dri-devel/2023-November/430875.html HTTP/1.1" 200 7388 "-" "TerraCotta https://github.com/CeramicTeam/CeramicTerracotta"
 2025-09-10 12:04:08 +02:00
dark-visitors
dd391bf960 Update from Dark Visitors 2025-09-10 00:53:56 +00:00
ai.robots.txt
229d1b4dbc Merge pull request #174 from karolyi/master 
Update Brightbot operator and details; add meta-webindexer entry
 2025-09-09 02:45:32 +00:00
Glyn Normington
4d506ca322
 		Merge pull request #174 from karolyi/master 
Update Brightbot operator and details; add meta-webindexer entry
 2025-09-09 03:45:23 +01:00
László Károlyi
ec508ab434
 		Update Brightbot operator and details; add meta-webindexer entry 
- Update Brightbot operator to https://brightdata.com/brightbot.
- Change Brightbot frequency to "At least one per minute."
- Expand Brightbot description with disguise tactics link.
- Add new entry for meta-webindexer under Meta operator.
- Set meta-webindexer respect to "Unclear at this time."
- Define meta-webindexer function as "AI Assistants."
- Set meta-webindexer frequency to "Unhinged, more than 1 per second."
- Include meta-webindexer description on improving Meta AI search.
 2025-09-08 21:42:06 +02:00
dark-visitors
0ed29412c9 Update from Dark Visitors v1.40 2025-08-28 00:55:47 +00:00
ai.robots.txt
2ad1c3e831 Merge pull request #168 from ai-robots-txt/google-firebase-shapbot 1.40 
chore: add Google-Firebase and ShapBot
 2025-08-27 10:13:17 +00:00
Glyn Normington
1677278c5a
 		Merge pull request #168 from ai-robots-txt/google-firebase-shapbot 
chore: add Google-Firebase and ShapBot
 2025-08-27 11:13:04 +01:00
Cory Dransfeldt
1a8edfa84a
 		chore: add Google-Firebase and ShapBot 2025-08-26 19:39:42 -07:00
dark-visitors
cf073d49f2 Update from Dark Visitors 2025-08-15 01:01:42 +00:00
ai.robots.txt
6d3f3e1712 Merge pull request #167 from ai-robots-txt/OpenAI-bot v1.39 
chore: adds OpenAI but user agent
 2025-08-14 01:45:05 +00:00
Glyn Normington
784b8440a5
 		Merge pull request #167 from ai-robots-txt/OpenAI-bot 
chore: adds OpenAI but user agent
 2025-08-14 02:44:53 +01:00
Cory Dransfeldt
0e687a5b58
 		chore: adds OpenAI but user agent 2025-08-13 09:01:55 -07:00
ai.robots.txt
ff9fc26404 Update from Dark Visitors 2025-08-01 01:12:53 +00:00
dark-visitors
146a229662 Update from Dark Visitors 2025-07-31 01:05:21 +00:00
ai.robots.txt
64f9d6ce9c Update from Dark Visitors 2025-07-30 01:05:21 +00:00
dark-visitors
085dd1071e Update from Dark Visitors 2025-07-29 01:11:26 +00:00
ai.robots.txt
9565c11d4c Merge pull request #166 from ai-robots-txt/summaly-bot 
fix: remove summaly
 2025-07-28 19:19:15 +00:00
Glyn Normington
8869442615
 		Merge pull request #166 from ai-robots-txt/summaly-bot 
fix: remove summaly
 2025-07-28 20:19:06 +01:00
ai.robots.txt
27420d6fed Merge pull request #164 from GlitzSmarter/main 
Adding YaK
 2025-07-28 18:29:57 +00:00
Glyn Normington
44e58a5ece
 		Merge pull request #164 from GlitzSmarter/main 
Adding YaK
 2025-07-28 19:29:48 +01:00
Glyn Normington
12c5368e04
 		Fix syntax 2025-07-28 19:29:19 +01:00