149 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
ai.robots.txt	3d4bf2c3db	restore original robots.json	2024-08-06 18:50:54 +00:00
ai.robots.txt	d6a5e8cd81	Removing previously generated files	2024-08-06 18:50:53 +00:00
Chenghao Mou	4cf82b703f	restore original robots.json	2024-08-06 19:50:38 +01:00
Chenghao Mou	0b6eba8dd5	skip push if no change	2024-08-06 19:41:38 +01:00
Chenghao Mou	379c339f97	skip push if no change	2024-08-06 19:41:38 +01:00
Chenghao Mou	01edb6c78c	Merge branch 'ai-robots-txt:main' into main	2024-08-06 19:35:03 +01:00
Chenghao Mou	2a3685385c	restrict scope	2024-08-06 19:33:49 +01:00
Cory Dransfeldt	85275e55b8	Merge pull request #31 from glyn/addfaq ... Add FAQ	2024-08-06 11:15:29 -07:00
Chenghao Mou	8c6482fb45	restore the cron	2024-08-06 18:12:41 +01:00
dark-visitors	63c7e742c3	Daily update from Dark Visitors	2024-08-06 16:54:29 +00:00
Chenghao Mou	55e92f4324	update existing ones	2024-08-06 17:48:06 +01:00
Chenghao Mou	52d54cf127	restore the cron	2024-08-06 17:28:07 +01:00
dark-visitors	fdd261dad4	Daily update from Dark Visitors	2024-08-06 16:27:02 +00:00
ai.robots.txt	6d2285f5e0	Add FAQ	2024-08-06 16:21:01 +00:00
ai.robots.txt	83d9397f17	Removing previously generated files	2024-08-06 16:21:00 +00:00
Glyn Normington	b4d25bf0cb	Add FAQ	2024-08-06 17:20:26 +01:00
Chenghao Mou	8ab1e30a6c	test workflow	2024-08-06 17:12:26 +01:00
Chenghao Mou	192bf67631	add dark visitor workflow	2024-08-06 17:02:23 +01:00
ai.robots.txt	e12ddc0f42	Merge pull request #29 from jbowdre/dev ... only build on changes to robots.json	2024-08-06 15:44:54 +00:00
ai.robots.txt	b54e274bbc	Removing previously generated files	2024-08-06 15:44:53 +00:00
Cory Dransfeldt	3e91a84d11	Merge pull request #29 from jbowdre/dev ... only build on changes to robots.json	2024-08-04 16:04:59 -07:00
John Bowdre	b0a93aeb70	only build on changes to robots.json	2024-08-04 17:45:18 -05:00
ai.robots.txt	eb924b9856	Merge pull request #28 from jsheard/patch-2 ... v1.13 Add Cloudflares first-party scraper blocking to FAQ	2024-08-04 21:54:17 +00:00
ai.robots.txt	1cfc071498	Removing previously generated files	2024-08-04 21:54:16 +00:00
Cory Dransfeldt	24c3509a6e	Merge pull request #28 from jsheard/patch-2 ... Add Cloudflares first-party scraper blocking to FAQ	2024-08-04 14:54:06 -07:00
ai.robots.txt	c2f177870f	Merge pull request #27 from jsheard/patch-1 ... Fix Imagesift user agent	2024-08-04 21:53:48 +00:00
ai.robots.txt	0072b8f5f0	Removing previously generated files	2024-08-04 21:53:47 +00:00
Cory Dransfeldt	9c7257e7cf	Merge pull request #27 from jsheard/patch-1 ... Fix Imagesift user agent	2024-08-04 14:53:36 -07:00
Joshua Sheard	8dbbdbf44c	Add Cloudflares first-party scraper blocking to FAQ	2024-08-04 21:38:02 +01:00
Joshua Sheard	146fd4ffba	Fix Imagesift user agent	2024-08-04 21:33:04 +01:00
ai.robots.txt	c7b781034e	chore: restore FriendlyCrawler + ImageSift v1.12	2024-08-04 19:29:01 +00:00
ai.robots.txt	9a8fa66772	Removing previously generated files	2024-08-04 19:29:00 +00:00
Cory Dransfeldt	1ca936ce11	chore: restore FriendlyCrawler + ImageSift	2024-08-04 12:28:48 -07:00
ai.robots.txt	8de5bc8e01	Merge pull request #25 from mirium999/add_icc_crawler ... Add ICC-Crawler	2024-08-04 01:21:56 +00:00
ai.robots.txt	8c632e1ba4	Removing previously generated files	2024-08-04 01:21:55 +00:00
Adam Newbold	8d4d52cdab	Merge pull request #25 from mirium999/add_icc_crawler ... Add ICC-Crawler	2024-08-03 21:21:45 -04:00
Mirium999	5826c18909	Add ICC-Crawler	2024-08-04 10:11:25 +09:00
ai.robots.txt	ffbad453f3	Merge pull request #24 from nisbet-hubbard/patch-5 ... Add last line of defence to FAQ	2024-08-03 14:27:47 +00:00
ai.robots.txt	b1907d86be	Removing previously generated files	2024-08-03 14:27:46 +00:00
Cory Dransfeldt	55c585e9e3	Merge pull request #24 from nisbet-hubbard/patch-5 ... Add last line of defence to FAQ	2024-08-03 07:27:37 -07:00
nisbet-hubbard	2b56c72bac	Update FAQ.md	2024-08-03 14:27:25 +08:00
nisbet-hubbard	b24e5cb3bb	Update FAQ.md	2024-08-03 14:12:50 +08:00
nisbet-hubbard	74b1502839	Update FAQ.md	2024-08-03 14:04:58 +08:00
ai.robots.txt	d8de1ebdd5	chore: contribution note	2024-08-02 16:32:00 +00:00
ai.robots.txt	9d8d3de8ed	Removing previously generated files	2024-08-02 16:31:59 +00:00
Cory Dransfeldt	349c35eed6	chore: contribution note	2024-08-02 09:31:48 -07:00
ai.robots.txt	b144225ece	chore: drop in additional data	2024-08-01 22:33:23 +00:00
ai.robots.txt	06b950bce9	Removing previously generated files	2024-08-01 22:33:23 +00:00
Cory Dransfeldt	b20dfec1e4	chore: drop in additional data	2024-08-01 15:33:07 -07:00
ai.robots.txt	f18f0d99b9	chore: remove test data	2024-08-01 22:29:02 +00:00