55 commits

Author	SHA1	Message	Date
Glyn Normington	c26c8c0911	Tidy README	2025-11-27 12:41:37 +00:00
Glyn Normington	b75163e796	Ensure Python3 is used	2025-11-27 12:38:12 +00:00
Glyn Normington	c3f2fe758e	Whitespace	2025-11-27 12:37:21 +00:00
Glyn Normington	7521c3af50	Fix link	2025-11-27 12:37:07 +00:00
Anshita-18H	4302fd1aca	Improve contributing section and fix formatting in README	2025-11-27 17:56:47 +05:30
Glyn Normington	c322d7852d	Merge pull request #130 from maxheadroom/main ... Create traefik-manual-setup.md	2025-11-26 12:08:11 +00:00
Glyn Normington	35588b2ddb	Typo.	2025-11-26 12:07:39 +00:00
Glyn Normington	6225e3e98e	Document how to ship a new release	2025-11-23 04:01:43 +00:00
nisbet-hubbard	c6e7d69dd5	Update README.md	2025-09-24 09:39:21 -07:00
Cory Dransfeldt	8b151b2cdc	Update README.md ... Co-authored-by: Glyn Normington <glyn.normington@gmail.com>	2025-05-21 06:52:36 -07:00
Cory Dransfeldt	8a8001cbec	chore(README): updates the opening line of our README to clarify the types of agents we block	2025-05-20 13:55:25 -07:00
Falko Zurell	684d11d889	moved traefik manual setup into docs ... moved the traefik manual setup into the docs directory and linked to it from the README.md	2025-05-16 13:50:58 +02:00
Florent Poinsaut	c66d180295	Merge branch 'main' into add-traefik-plugin	2025-05-14 22:06:56 +02:00
Florent Poinsaut	b4610a725c	Add Traefik plugin	2025-05-14 14:11:56 +02:00
Cory Dransfeldt	36a52a88d8	Bing AI opt-out instructions	2025-05-12 20:20:18 -07:00
Crazyroostereye	1310dbae46	Added a Caddyfile converter (#110) ... Co-authored-by: Julian Beittel <julian@beittel.net> Co-authored-by: Glyn Normington <work@underlap.org>	2025-05-01 11:21:32 +01:00
Rik Wijnen	66da70905f	Fixed incorrect English sentence.	2025-04-28 09:09:40 +02:00
Rik Wijnen	50e739dd73	HAProxy converter added.	2025-04-28 08:51:02 +02:00
Thomas Leister	5a312c5f4d	Mention Nginx config feature in README	2025-03-27 12:43:29 +01:00
Glyn Normington	1d55a205e4	Document testing in README ... Fixes: https://github.com/ai-robots-txt/ai.robots.txt/issues/81	2025-02-18 16:49:08 +00:00
always-be-testing	5b13c2e504	add more concise message about verified bots ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-02-15 11:22:10 -05:00
always-be-testing	af87b85d7f	include return after heading	2025-02-14 12:39:08 -05:00
always-be-testing	f99339922f	grammar update and include syntax for verified bot condition	2025-02-14 12:36:33 -05:00
always-be-testing	e396a2ec78	forgot to include heading	2025-02-14 12:31:20 -05:00
always-be-testing	261a2b83b9	update README to inclide list of ai bots Cloudflare considers verified	2025-02-14 12:26:19 -05:00
Massimo Gismondi	a9956f7825	Removed additional sections	2025-01-20 06:50:48 +01:00
Massimo Gismondi	33c38ee70b	Update README.md ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-01-20 06:28:32 +01:00
Massimo Gismondi	52241bdca6	Update README.md ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-01-20 06:27:56 +01:00
Massimo Gismondi	013b7abfa1	Update README.md ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-01-20 06:27:02 +01:00
Massimo Gismondi	70fd6c0fb1	Add mention of htaccess in readme ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-01-20 06:25:07 +01:00
Massimo Gismondi	d65128d10a	Removed paragraph in favour of future FAQ.md ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-01-18 12:41:09 +01:00
Massimo Gismondi	1cc4b59dfc	Shortened htaccess instructions ... Co-authored-by: Glyn Normington <work@underlap.org>	2025-01-18 12:40:03 +01:00
Massimo Gismondi	b455af66e7	Adding clarification about performance and code comment	2025-01-17 21:42:08 +01:00
Massimo Gismondi	189e75bbfd	Adding usage instructions	2025-01-17 21:25:23 +01:00
Glyn Normington	75343c790e	Merge pull request #38 from urvish-p80/main ... Add an additional resource - README.md	2024-09-27 01:26:04 +01:00
Julian Mair	af05890b07	Update README.md ... For people who don't use or don't want to use RSS for this, I've added a little explanation of how to subscribe to releases via GitHub.	2024-09-23 23:27:27 +02:00
Urvish Patel	0106d4b15a	Add additional resource - README.md ... A detailed blogpost to - See the live dashboard showing the websites that are blocking AI Bots such as GPTBot, CCBot, Google-extended and ByteSpider from crawling and scraping the content on their website. Learn which AI crawlers / scrapers do what and how to block them using Robots.txt.	2024-09-23 08:19:27 -04:00
Cory Dransfeldt	1d3194f75d	chore: update readme	2024-08-18 11:34:43 -07:00
nisbet-hubbard	46540633ba	Update README.md	2024-08-10 08:22:28 +08:00
Cory Dransfeldt	349c35eed6	chore: contribution note	2024-08-02 09:31:48 -07:00
Glyn Normington	c17cae6e9d	link to bot metrics table ... Make it easier to view the table.	2024-07-17 02:28:32 +01:00
Glyn Normington	4a8ce0b51d	Add reference ... This links to https://github.com/glyn/nginx_robot_access	2024-06-29 12:20:38 +01:00
Corbin Davenport	20dc4e16ad	Update releases feed links	2024-06-21 22:02:54 -04:00
Cory Dransfeldt	a980ace544	chore: update readme	2024-06-20 08:11:38 -07:00
Cory Dransfeldt	a90ee5e9f0	chore: clean up bots and narrow scope	2024-06-20 08:09:21 -07:00
Cory Dransfeldt	1a4868f1a2	chore: update readme	2024-06-12 19:05:15 -07:00
Cory Dransfeldt	b5d4d5bc74	chore: update readme	2024-06-12 19:01:23 -07:00
Cory Dransfeldt	0d9b75b227	chore: move additional resources to wiki	2024-04-12 09:33:19 -07:00
Cory Dransfeldt	ef5d847e65	chore: add gitignore; move logo file	2024-04-10 10:44:07 -07:00
Matthew Linares	048045ac6d	Adds table of bot details and metrics.	2024-04-08 20:29:45 +01:00