remote/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2026-06-03 23:56:55 +02:00
Code Issues Projects Releases Packages Wiki Activity

Update with lighttpd instructions.

Browse source
This commit is contained in:
Flynn Marquardt 2026-02-13 19:20:12 +01:00
commit deae8eed2d
1 changed files with 2 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -16,6 +16,7 @@ This repository provides the following files:
- `nginx-block-ai-bots.conf`
- `Caddyfile`
- `haproxy-block-ai-bots.txt`
- `lighttpd-block-ai-bots.conf`
`robots.txt` implements the Robots Exclusion Protocol ([RFC 9309](https://www.rfc-editor.org/rfc/rfc9309.html)).
@ -35,6 +36,7 @@ Note that, as stated in the [httpd documentation](https://httpd.apache.org/docs/
```
(Note that the path of the `haproxy-block-ai-bots.txt` may be different in your environment.)
`lighttpd-block-ai-bots.conf` can be included with `include "fragments/lighttpd-block-ai-bots.conf"` in your lighttpd configuratio either global or in conditional section.
[Bing uses the data it crawls for AI and training, you may opt out by adding a `meta` tag to the `head` of your site.](./docs/additional-steps/bing.md)