615 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Glyn Normington	243ec6b67d	Merge pull request #216 from flymarq/lighttpd ... v1.45 Lighttpd	2026-02-17 17:28:34 +00:00
Flynn Marquardt	793b0454f2	Use adverb.	2026-02-15 11:20:34 +01:00
Flynn Marquardt	790bff1817	Add reference file for lighttpd test.	2026-02-15 11:18:35 +01:00
Flynn Marquardt	742f9b9b5e	Add lighttpd test.	2026-02-15 11:17:54 +01:00
ai.robots.txt	48bfefbc5f	Update from Dark Visitors	2026-02-14 01:16:06 +00:00
Flynn Marquardt	5f0dd97ccb	Add lighttpd sample file.	2026-02-13 19:23:19 +01:00
Flynn Marquardt	55d404c115	Fix typos.	2026-02-13 19:21:31 +01:00
Flynn Marquardt	deae8eed2d	Update with lighttpd instructions.	2026-02-13 19:20:12 +01:00
Flynn Marquardt	70e83e6056	Add generation and output of lighttpd configuration fragment.	2026-02-13 19:18:46 +01:00
dark-visitors	a762015e58	Update from Dark Visitors	2026-02-13 01:23:17 +00:00
ai.robots.txt	aa8519ec10	Update from Dark Visitors	2025-12-21 01:07:06 +00:00
dark-visitors	83485effdb	Update from Dark Visitors	2025-12-20 00:58:49 +00:00
ai.robots.txt	8b8bf9da5d	Update from Dark Visitors	2025-12-06 00:58:16 +00:00
dark-visitors	f1c752ef12	Update from Dark Visitors	2025-12-05 01:00:44 +00:00
Adam Newbold	51afa7113a	Update ai_robots_update.yml with rebase command to fix scheduled run	2025-12-03 20:15:10 -05:00
ai.robots.txt	7598d77e4a	Update from Dark Visitors	2025-12-04 01:10:02 +00:00
dark-visitors	45b071b29f	Update from Dark Visitors	2025-12-04 01:00:27 +00:00
dark-visitors	f61b3496f7	Update from Dark Visitors	2025-12-03 01:00:32 +00:00
dark-visitors	8363d4fdd4	Update from Dark Visitors	2025-12-02 01:25:24 +00:00
Adam Newbold	2ccd443581	Update ai_robots_update.yml with workflow_dispatch ... Adding workflow_dispatch to enable manual triggers of this schedule job (for testing)	2025-12-01 20:24:41 -05:00
Adam Newbold	6d75f3c1c9	Update robots.py to address error on line 57 ... Attempting to work around an error that prevents parsing the Dark Visitors site	2025-12-01 20:18:29 -05:00
Glyn Normington	56010ef913	Merge pull request #205 from fiskhandlarn/fix/editorconfig ... Fix/editorconfig	2025-11-29 10:02:08 +00:00
ai.robots.txt	3fadc88a23	Merge pull request #206 from newbold/main ... Adding LAIONDownloader	2025-11-29 10:00:52 +00:00
Glyn Normington	47c077a8ef	Merge pull request #206 from newbold/main ... Adding LAIONDownloader	2025-11-29 10:00:42 +00:00
Adam Newbold	f5d7ccb243	Fixed invalid JSON	2025-11-28 14:10:38 -05:00
Adam Newbold	30d719a09a	Adding LAIONDownloader	2025-11-28 14:08:18 -05:00
fiskhandlarn	05bbdebeaa	feat: disallow final newline for files generated by python ... if any of these files have ending newlines the tests will fail	2025-11-28 10:47:22 +01:00
fiskhandlarn	c6ce9329a1	fix: ensure whitespace as defined in .editorconfig	2025-11-28 10:46:22 +01:00
ai.robots.txt	10d5ae2870	Merge pull request #200 from glyn/deepseek ... Clarify that DeepSeekBot does not respect robots.txt	2025-11-27 14:00:50 +00:00
Glyn Normington	4467002298	Merge pull request #200 from glyn/deepseek ... Clarify that DeepSeekBot does not respect robots.txt	2025-11-27 14:00:40 +00:00
Glyn Normington	4a159a818f	Merge pull request #199 from glyn/editorconfig ... Standardise editor options	2025-11-27 13:54:03 +00:00
Glyn Normington	3d6b33a71a	Merge pull request #202 from glyn/formatting ... Tidy README	2025-11-27 12:42:31 +00:00
Glyn Normington	c26c8c0911	Tidy README	2025-11-27 12:41:37 +00:00
Glyn Normington	91959fe791	Merge pull request #201 from Anshita-18H/add-requirements-file ... Add requirements.txt with project dependencies	2025-11-27 12:38:28 +00:00
Glyn Normington	b75163e796	Ensure Python3 is used	2025-11-27 12:38:12 +00:00
Glyn Normington	f46754d280	Order deps	2025-11-27 12:37:35 +00:00
Glyn Normington	c3f2fe758e	Whitespace	2025-11-27 12:37:21 +00:00
Glyn Normington	7521c3af50	Fix link	2025-11-27 12:37:07 +00:00
Anshita-18H	4302fd1aca	Improve contributing section and fix formatting in README	2025-11-27 17:56:47 +05:30
Anshita-18H	9ca5033927	Deduplicate requirements.txt and add installation instructions to README	2025-11-27 17:44:47 +05:30
Anshita-18H	9cc3dbc05f	Add requirements.txt with project dependencies	2025-11-27 17:06:59 +05:30
Glyn Normington	c322d7852d	Merge pull request #130 from maxheadroom/main ... Create traefik-manual-setup.md	2025-11-26 12:08:11 +00:00
Glyn Normington	35588b2ddb	Typo.	2025-11-26 12:07:39 +00:00
Glyn Normington	ab24e41106	Clarify that DeepSeekBot does not respect robots.txt ... Fixes https://github.com/ai-robots-txt/ai.robots.txt/issues/198	2025-11-26 12:02:42 +00:00
Glyn Normington	b681c0c0d8	Standardise editor options ... This was motivated by @fiskhandlarn's comment: https://github.com/ai-robots-txt/ai.robots.txt/pull/195#issuecomment-3576331322	2025-11-26 11:54:10 +00:00
Glyn Normington	4e7e28335f	Merge pull request #195 from fiskhandlarn/patch-2 ... feat: allow robots access to `/robots.txt` in nginx	2025-11-26 11:50:15 +00:00
fiskhandlarn	a6cf6b204b	test: update test nginx conf	2025-11-25 16:47:04 +01:00
fiskhandlarn	2679fcad34	feat: update nginx generator	2025-11-25 16:46:31 +01:00
fiskhandlarn	ef8eda4fe6	chore: normalize quote style	2025-11-25 16:39:35 +01:00
Adam Newbold	a29102f0fc	Merge pull request #196 from glyn/releasing ... v1.44 Document how to ship a new release	2025-11-25 10:34:25 -05:00