remote/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2026-05-30 05:36:56 +02:00
Code Issues Projects Releases Packages Wiki Activity

Commit graph

  • ee3a96ee5c Update from Dark Visitors main dark-visitors 2026-05-22 02:33:30 +00:00
  • 1fbf7a06ac
         Merge pull request #231 from cfcryptotrader-rgb/cfcryptotrader-sync-generated-robots v1.46 Glyn Normington 2026-05-13 07:44:48 +01:00
  • 8d5a081a16 Keep generated robot files in dark visitors update cfcryptotrader-rgb 2026-05-13 00:44:06 -03:00
  • aea7db9e34 Update from Dark Visitors dark-visitors 2026-05-13 02:24:20 +00:00
  • 767fcc6f15
         Merge pull request #229 from axeleroy/main Glyn Normington 2026-05-12 04:33:19 +01:00
  • 6adc1f7b53
         Add missing bot categories Axel Leroy 2026-05-11 15:31:22 +02:00
  • 851ce068cf Update from Dark Visitors dark-visitors 2026-05-03 02:01:44 +00:00
  • 8b0bc322f4 Merge pull request #227 from gsauthof/naget-news ai.robots.txt 2026-05-02 15:05:09 +00:00
  • 7244a7e870
         Merge pull request #227 from gsauthof/naget-news Glyn Normington 2026-05-02 16:04:59 +01:00
  • a9f686064f Add NagetBot and newsai user-agents Georg Sauthoff 2026-05-02 13:04:51 +02:00
  • 198653b59a Update from Dark Visitors ai.robots.txt 2026-03-27 01:26:26 +00:00
  • ad47cafc97 Update from Dark Visitors dark-visitors 2026-03-26 01:26:23 +00:00
  • 2d2ec4ce8d Update from Dark Visitors dark-visitors 2026-03-25 01:20:42 +00:00
  • dc0e28989b tests: make nginx bypass assertion explicit shuofengzhang 2026-03-21 19:40:35 +00:00
  • 507d16262a tests: make converter-helper coverage independent and CI-executed shuofengzhang 2026-03-18 19:30:27 +00:00
  • d32f959fea test: reduce converter test circularity and overlap shuofengzhang 2026-03-16 19:24:54 +00:00
  • 0c995a065e Merge remote-tracking branch 'origin/main' into add-tests-converters shuofengzhang 2026-03-15 13:40:47 +00:00
  • c223c8bced chore: suppress expected robot-name normalization noise shuofengzhang 2026-03-15 13:34:40 +00:00
  • e7f8b0ca91 ci: run pytest suite for converter tests shuofengzhang 2026-03-09 13:58:08 +08:00
  • ee50da4b4e test: broaden converter and file update coverage shuofengzhang 2026-03-09 08:51:22 +08:00
  • f8bd13d9a3 test: expand converter coverage for output formats shuofengzhang 2026-03-09 05:57:04 +08:00
  • 86d582b11c Update from Dark Visitors ai.robots.txt 2026-03-07 01:12:48 +00:00
  • 3de8865f00 Update from Dark Visitors dark-visitors 2026-03-06 01:21:01 +00:00
  • e73aacfd22 Add tests for converter helpers shuofengzhang 2026-03-03 20:35:19 +08:00
  • 243ec6b67d
         Merge pull request #216 from flymarq/lighttpd v1.45 Glyn Normington 2026-02-17 17:28:34 +00:00
  • 793b0454f2 Use adverb. Flynn Marquardt 2026-02-15 11:20:34 +01:00
  • 790bff1817 Add reference file for lighttpd test. Flynn Marquardt 2026-02-15 11:18:35 +01:00
  • 742f9b9b5e Add lighttpd test. Flynn Marquardt 2026-02-15 11:17:54 +01:00
  • 48bfefbc5f Update from Dark Visitors ai.robots.txt 2026-02-14 01:16:06 +00:00
  • 5f0dd97ccb Add lighttpd sample file. Flynn Marquardt 2026-02-13 19:23:19 +01:00
  • 55d404c115 Fix typos. Flynn Marquardt 2026-02-13 19:21:31 +01:00
  • deae8eed2d Update with lighttpd instructions. Flynn Marquardt 2026-02-13 19:20:12 +01:00
  • 70e83e6056 Add generation and output of lighttpd configuration fragment. Flynn Marquardt 2026-02-13 19:18:46 +01:00
  • a762015e58 Update from Dark Visitors dark-visitors 2026-02-13 01:23:17 +00:00
  • 03c6890c45
         Rename README.mdf to README.md جواد 2026-01-03 20:08:09 +03:00
  • aa8519ec10 Update from Dark Visitors ai.robots.txt 2025-12-21 01:07:06 +00:00
  • 83485effdb Update from Dark Visitors dark-visitors 2025-12-20 00:58:49 +00:00
  • 8b8bf9da5d Update from Dark Visitors ai.robots.txt 2025-12-06 00:58:16 +00:00
  • f1c752ef12 Update from Dark Visitors dark-visitors 2025-12-05 01:00:44 +00:00
  • 51afa7113a
         Update ai_robots_update.yml with rebase command to fix scheduled run Adam Newbold 2025-12-03 20:15:10 -05:00
  • 7598d77e4a Update from Dark Visitors ai.robots.txt 2025-12-04 01:10:02 +00:00
  • 45b071b29f Update from Dark Visitors dark-visitors 2025-12-04 01:00:27 +00:00
  • f61b3496f7 Update from Dark Visitors dark-visitors 2025-12-03 01:00:32 +00:00
  • 8363d4fdd4 Update from Dark Visitors dark-visitors 2025-12-02 01:25:24 +00:00
  • 2ccd443581
         Update ai_robots_update.yml with workflow_dispatch Adam Newbold 2025-12-01 20:24:41 -05:00
  • 6d75f3c1c9
         Update robots.py to address error on line 57 Adam Newbold 2025-12-01 20:18:29 -05:00
  • 56010ef913
         Merge pull request #205 from fiskhandlarn/fix/editorconfig Glyn Normington 2025-11-29 10:02:08 +00:00
  • 3fadc88a23 Merge pull request #206 from newbold/main ai.robots.txt 2025-11-29 10:00:52 +00:00
  • 47c077a8ef
         Merge pull request #206 from newbold/main Glyn Normington 2025-11-29 10:00:42 +00:00
  • f5d7ccb243
         Fixed invalid JSON Adam Newbold 2025-11-28 14:10:38 -05:00
  • 30d719a09a
         Adding LAIONDownloader Adam Newbold 2025-11-28 14:08:18 -05:00
  • 05bbdebeaa feat: disallow final newline for files generated by python fiskhandlarn 2025-11-28 10:47:22 +01:00
  • c6ce9329a1 fix: ensure whitespace as defined in .editorconfig fiskhandlarn 2025-11-28 10:46:22 +01:00
  • 7d6827bc72 fix: generate requirements.txt for this repo only fiskhandlarn 2025-11-28 10:30:17 +01:00
  • 10d5ae2870 Merge pull request #200 from glyn/deepseek ai.robots.txt 2025-11-27 14:00:50 +00:00
  • 4467002298
         Merge pull request #200 from glyn/deepseek Glyn Normington 2025-11-27 14:00:40 +00:00
  • 4a159a818f
         Merge pull request #199 from glyn/editorconfig Glyn Normington 2025-11-27 13:54:03 +00:00
  • 3d6b33a71a
         Merge pull request #202 from glyn/formatting Glyn Normington 2025-11-27 12:42:31 +00:00
  • c26c8c0911 Tidy README Glyn Normington 2025-11-27 12:41:37 +00:00
  • 91959fe791
         Merge pull request #201 from Anshita-18H/add-requirements-file Glyn Normington 2025-11-27 12:38:28 +00:00
  • b75163e796
         Ensure Python3 is used Glyn Normington 2025-11-27 12:38:12 +00:00
  • f46754d280
         Order deps Glyn Normington 2025-11-27 12:37:35 +00:00
  • c3f2fe758e
         Whitespace Glyn Normington 2025-11-27 12:37:21 +00:00
  • 7521c3af50
         Fix link Glyn Normington 2025-11-27 12:37:07 +00:00
  • 4302fd1aca Improve contributing section and fix formatting in README Anshita-18H 2025-11-27 17:56:47 +05:30
  • 9ca5033927 Deduplicate requirements.txt and add installation instructions to README Anshita-18H 2025-11-27 17:44:47 +05:30
  • 9cc3dbc05f Add requirements.txt with project dependencies Anshita-18H 2025-11-27 17:06:59 +05:30
  • c322d7852d
         Merge pull request #130 from maxheadroom/main Glyn Normington 2025-11-26 12:08:11 +00:00
  • 35588b2ddb
         Typo. Glyn Normington 2025-11-26 12:07:39 +00:00
  • ab24e41106 Clarify that DeepSeekBot does not respect robots.txt Glyn Normington 2025-11-26 12:02:42 +00:00
  • b681c0c0d8 Standardise editor options Glyn Normington 2025-11-26 11:54:10 +00:00
  • 4e7e28335f
         Merge pull request #195 from fiskhandlarn/patch-2 Glyn Normington 2025-11-26 11:50:15 +00:00
  • a6cf6b204b test: update test nginx conf fiskhandlarn 2025-11-25 16:47:04 +01:00
  • 2679fcad34 feat: update nginx generator fiskhandlarn 2025-11-25 16:46:31 +01:00
  • ef8eda4fe6 chore: normalize quote style fiskhandlarn 2025-11-25 16:39:35 +01:00
  • a29102f0fc
         Merge pull request #196 from glyn/releasing v1.44 Adam Newbold 2025-11-25 10:34:25 -05:00
  • 0b3266b35f feat: allow robots access to /robots.txt in nginx fiskhandlarn 2025-11-20 11:50:37 +01:00
  • 6225e3e98e Document how to ship a new release Glyn Normington 2025-11-23 04:01:43 +00:00
  • be4d74412c Merge pull request #194 from glyn/193-kendra ai.robots.txt 2025-11-21 16:32:11 +00:00
  • 729be4693a
         Merge pull request #194 from glyn/193-kendra Cory Dransfeldt 2025-11-21 08:32:01 -08:00
  • efb4d260da delete extraneous hyphen Glyn Normington 2025-11-20 10:23:09 +00:00
  • e2726ac160 Merge pull request #192 from ai-robots-txt/cdransf/notebooklm-klaviyo v1.43 ai.robots.txt 2025-11-14 21:01:25 +00:00
  • 663d030f96
         Merge pull request #192 from ai-robots-txt/cdransf/notebooklm-klaviyo Glyn Normington 2025-11-14 21:01:13 +00:00
  • 28b45ea08d
         chore: adds NotebookLM and KlaviyoAIBot agents Cory Dransfeldt 2025-11-14 11:03:03 -08:00
  • 443dd27527 Merge pull request #189 from ai-robots-txt/cdransf/atlassian-amazon-bots ai.robots.txt 2025-11-05 23:50:59 +00:00
  • 60b6a0829d
         Merge pull request #189 from ai-robots-txt/cdransf/atlassian-amazon-bots Glyn Normington 2025-11-05 23:50:48 +00:00
  • 00bf2b0e13
         chore(robots.json): add AmazonBuyForMe and atlassian-bot Cory Dransfeldt 2025-11-05 14:02:37 -08:00
  • e87eb706e3 Merge pull request #188 from ai-robots-txt/cdransf/buddybot ai.robots.txt 2025-11-04 05:16:28 +00:00
  • 3d41350256
         Merge pull request #188 from ai-robots-txt/cdransf/buddybot Glyn Normington 2025-11-04 05:16:15 +00:00
  • 808451055c
         chore: adds BuddyBot Cory Dransfeldt 2025-11-03 13:39:07 -08:00
  • 5cad0ee389 Merge pull request #187 from ai-robots-txt/cdransf/add-Linguee-Bot ai.robots.txt 2025-10-24 18:05:01 +00:00
  • d55c9980cd
         Merge pull request #187 from ai-robots-txt/cdransf/add-Linguee-Bot Glyn Normington 2025-10-24 19:04:42 +01:00
  • 192b0a2eef
         chore: adds Linguee Bot Cory Dransfeldt 2025-10-24 10:35:04 -07:00
  • 97e19445ce Update from Dark Visitors dark-visitors 2025-10-24 00:53:30 +00:00
  • 0bc2361be8 Merge pull request #186 from ai-robots-txt/third-party ai.robots.txt 2025-10-21 15:23:46 +00:00
  • 511d8c955d
         Merge pull request #186 from ai-robots-txt/third-party Cory Dransfeldt 2025-10-21 08:23:28 -07:00
  • b89a9eae6a Use third party in text, rather than first third-party Glyn Normington 2025-10-21 01:11:32 +01:00
  • 646ab08e15 Merge pull request #185 from ai-robots-txt/cdransf/add-IbouBot ai.robots.txt 2025-10-21 00:09:24 +00:00
  • 5c3da1c1af
         Merge pull request #185 from ai-robots-txt/cdransf/add-IbouBot Glyn Normington 2025-10-21 01:09:06 +01:00
  • d22b8dfd7b
         Description is third party, not first Glyn Normington 2025-10-21 01:08:42 +01:00