remote/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2026-09-11 10:34:05 +02:00
Code Issues Projects Releases Packages Wiki Activity

docs(README): add Bot Ledger to Related resources

Browse source 
Free, static directory of verified AI crawlers with a one-click robots.txt/llms.txt generator, thematically aligned with this project's goal of helping site owners manage AI crawler access.
This commit is contained in:
Dan 2026-08-07 08:12:59 -06:00
commit abe839e4e0
1 changed files with 3 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -47,6 +47,9 @@ middleware plugin for [Traefik](https://traefik.io/traefik/) to automatically ad
file on-the-fly.
- Alternatively you can [manually configure Traefik](./docs/traefik-manual-setup.md) to centrally serve a static `robots.txt`.
- [Bot Ledger](https://farrelldan.github.io/ai-bot-directory/): free, static directory of verified AI crawlers with a one-click `robots.txt` and `llms.txt` generator. No signup required.
## Contributing
A note about contributing: updates should be added/made to `robots.json`. A GitHub action will then generate the updated `robots.txt`, `table-of-bot-metrics.md`, `.htaccess` and `nginx-block-ai-bots.conf`.