589 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Adam Newbold	f5d7ccb243	Fixed invalid JSON	2025-11-28 14:10:38 -05:00
Adam Newbold	30d719a09a	Adding LAIONDownloader	2025-11-28 14:08:18 -05:00
ai.robots.txt	10d5ae2870	Merge pull request #200 from glyn/deepseek ... Clarify that DeepSeekBot does not respect robots.txt	2025-11-27 14:00:50 +00:00
Glyn Normington	4467002298	Merge pull request #200 from glyn/deepseek ... Clarify that DeepSeekBot does not respect robots.txt	2025-11-27 14:00:40 +00:00
Glyn Normington	4a159a818f	Merge pull request #199 from glyn/editorconfig ... Standardise editor options	2025-11-27 13:54:03 +00:00
Glyn Normington	3d6b33a71a	Merge pull request #202 from glyn/formatting ... Tidy README	2025-11-27 12:42:31 +00:00
Glyn Normington	c26c8c0911	Tidy README	2025-11-27 12:41:37 +00:00
Glyn Normington	91959fe791	Merge pull request #201 from Anshita-18H/add-requirements-file ... Add requirements.txt with project dependencies	2025-11-27 12:38:28 +00:00
Glyn Normington	b75163e796	Ensure Python3 is used	2025-11-27 12:38:12 +00:00
Glyn Normington	f46754d280	Order deps	2025-11-27 12:37:35 +00:00
Glyn Normington	c3f2fe758e	Whitespace	2025-11-27 12:37:21 +00:00
Glyn Normington	7521c3af50	Fix link	2025-11-27 12:37:07 +00:00
Anshita-18H	4302fd1aca	Improve contributing section and fix formatting in README	2025-11-27 17:56:47 +05:30
Anshita-18H	9ca5033927	Deduplicate requirements.txt and add installation instructions to README	2025-11-27 17:44:47 +05:30
Anshita-18H	9cc3dbc05f	Add requirements.txt with project dependencies	2025-11-27 17:06:59 +05:30
Glyn Normington	c322d7852d	Merge pull request #130 from maxheadroom/main ... Create traefik-manual-setup.md	2025-11-26 12:08:11 +00:00
Glyn Normington	35588b2ddb	Typo.	2025-11-26 12:07:39 +00:00
Glyn Normington	ab24e41106	Clarify that DeepSeekBot does not respect robots.txt ... Fixes https://github.com/ai-robots-txt/ai.robots.txt/issues/198	2025-11-26 12:02:42 +00:00
Glyn Normington	b681c0c0d8	Standardise editor options ... This was motivated by @fiskhandlarn's comment: https://github.com/ai-robots-txt/ai.robots.txt/pull/195#issuecomment-3576331322	2025-11-26 11:54:10 +00:00
Glyn Normington	4e7e28335f	Merge pull request #195 from fiskhandlarn/patch-2 ... feat: allow robots access to `/robots.txt` in nginx	2025-11-26 11:50:15 +00:00
fiskhandlarn	a6cf6b204b	test: update test nginx conf	2025-11-25 16:47:04 +01:00
fiskhandlarn	2679fcad34	feat: update nginx generator	2025-11-25 16:46:31 +01:00
fiskhandlarn	ef8eda4fe6	chore: normalize quote style	2025-11-25 16:39:35 +01:00
Adam Newbold	a29102f0fc	Merge pull request #196 from glyn/releasing ... v1.44 Document how to ship a new release	2025-11-25 10:34:25 -05:00
fiskhandlarn	0b3266b35f	feat: allow robots access to /robots.txt in nginx	2025-11-25 16:22:14 +01:00
Glyn Normington	6225e3e98e	Document how to ship a new release	2025-11-23 04:01:43 +00:00
ai.robots.txt	be4d74412c	Merge pull request #194 from glyn/193-kendra ... delete extraneous hyphen	2025-11-21 16:32:11 +00:00
Cory Dransfeldt	729be4693a	Merge pull request #194 from glyn/193-kendra ... delete extraneous hyphen	2025-11-21 08:32:01 -08:00
Glyn Normington	efb4d260da	delete extraneous hyphen ... Fixes https://github.com/ai-robots-txt/ai.robots.txt/issues/193	2025-11-20 10:23:48 +00:00
ai.robots.txt	e2726ac160	Merge pull request #192 from ai-robots-txt/cdransf/notebooklm-klaviyo ... v1.43 chore: adds NotebookLM and KlaviyoAIBot agents	2025-11-14 21:01:25 +00:00
Glyn Normington	663d030f96	Merge pull request #192 from ai-robots-txt/cdransf/notebooklm-klaviyo ... chore: adds NotebookLM and KlaviyoAIBot agents	2025-11-14 21:01:13 +00:00
Cory Dransfeldt	28b45ea08d	chore: adds NotebookLM and KlaviyoAIBot agents	2025-11-14 11:03:03 -08:00
ai.robots.txt	443dd27527	Merge pull request #189 from ai-robots-txt/cdransf/atlassian-amazon-bots ... chore(robots.json): add AmazonBuyForMe and atlassian-bot	2025-11-05 23:50:59 +00:00
Glyn Normington	60b6a0829d	Merge pull request #189 from ai-robots-txt/cdransf/atlassian-amazon-bots ... chore(robots.json): add AmazonBuyForMe and atlassian-bot	2025-11-05 23:50:48 +00:00
Cory Dransfeldt	00bf2b0e13	chore(robots.json): add AmazonBuyForMe and atlassian-bot	2025-11-05 14:02:37 -08:00
ai.robots.txt	e87eb706e3	Merge pull request #188 from ai-robots-txt/cdransf/buddybot ... chore: adds BuddyBot	2025-11-04 05:16:28 +00:00
Glyn Normington	3d41350256	Merge pull request #188 from ai-robots-txt/cdransf/buddybot ... chore: adds BuddyBot	2025-11-04 05:16:15 +00:00
Cory Dransfeldt	808451055c	chore: adds BuddyBot	2025-11-03 13:39:07 -08:00
ai.robots.txt	5cad0ee389	Merge pull request #187 from ai-robots-txt/cdransf/add-Linguee-Bot ... chore: adds Linguee Bot	2025-10-24 18:05:01 +00:00
Glyn Normington	d55c9980cd	Merge pull request #187 from ai-robots-txt/cdransf/add-Linguee-Bot ... chore: adds Linguee Bot	2025-10-24 19:04:42 +01:00
Cory Dransfeldt	192b0a2eef	chore: adds Linguee Bot	2025-10-24 10:35:04 -07:00
dark-visitors	97e19445ce	Update from Dark Visitors	2025-10-24 00:53:30 +00:00
ai.robots.txt	0bc2361be8	Merge pull request #186 from ai-robots-txt/third-party ... Use third party in text, rather than first	2025-10-21 15:23:46 +00:00
Cory Dransfeldt	511d8c955d	Merge pull request #186 from ai-robots-txt/third-party ... Use third party in text, rather than first	2025-10-21 08:23:28 -07:00
Glyn Normington	b89a9eae6a	Use third party in text, rather than first	2025-10-21 01:11:32 +01:00
ai.robots.txt	646ab08e15	Merge pull request #185 from ai-robots-txt/cdransf/add-IbouBot ... chore: adds IbouBot	2025-10-21 00:09:24 +00:00
Glyn Normington	5c3da1c1af	Merge pull request #185 from ai-robots-txt/cdransf/add-IbouBot ... chore: adds IbouBot	2025-10-21 01:09:06 +01:00
Glyn Normington	d22b8dfd7b	Description is third party, not first	2025-10-21 01:08:42 +01:00
Cory Dransfeldt	19c1d346c3	chore: adds IbouBot	2025-10-20 09:28:31 -07:00
ai.robots.txt	2fa0e9119c	Merge pull request #183 from ai-robots-txt/cdransf/bot-additions-cloudflare ... v1.42 chore: add amazon-kendra-, Anomura, Cloudflare-AutoRAG and Bravebot	2025-10-20 14:34:43 +00:00