46 commits

Author	SHA1	Message	Date
Özden und Julia	4021a10239	Pass commit messages to bash via env, not template interpolation ... `git commit -m "${{ github.event.head_commit.message }}"` splices arbitrary text straight into a double-quoted bash string. A message containing a double quote closes the string early and the remainder is re-parsed as shell words. This is what broke main after #248 merged. That PR's title contained "Unclear at this time." (with quotes), so the merge commit message did too, and the line bash actually ran was: git commit -m "Fill "Unclear at this time." from primary operator docs ..." which git received as: -m "Fill Unclear" at this "time. from primary operator docs ..." Hence `error: pathspec 'at' did not match any file(s) known to git`, a failed run, and the revert in 80c19fc. The data in that PR was fine — robots.py --convert exits 0 against it and code/tests.py passes 13/13. The same interpolation is also a script-injection vector, which is the more important reason to change it: a PR title is attacker-controlled, and chore: tidy"; <any command>; echo " executes that command on the runner with the workflow's token. `inputs.message` has the same shape in the `if [ -n ... ]` test and its own `git commit -m`, so all three are moved. Passing through `env:` and quoting the shell variable is GitHub's documented recommendation for untrusted values. The variable is expanded by bash after parsing, so quotes, newlines and `$(...)` stay literal text. Verified: code/tests.py 13/13 · python code/robots.py --convert exits 0 and leaves robots.txt, table-of-bot-metrics.md and every server-config output byte-identical · reproduced both the parse failure and the injection locally against the old form, and confirmed the env form commits the same message verbatim, quotes included.	2026-08-04 09:09:10 +02:00
Glyn Normington	5065ca6055	Update .github/workflows/ai_robots_update.yml ... For consistency.	2026-07-09 01:39:50 +01:00
Gavin King	a35e6f9a09	Update ai_robots_update.yml	2026-07-08 20:23:53 -04:00
Gavin King	c43c5afd07	More name changes	2026-07-08 15:43:07 -04:00
Gavin King	0c5de5b90b	Updating the name	2026-07-08 15:30:25 -04:00
cfcryptotrader-rgb	8d5a081a16	Keep generated robot files in dark visitors update	2026-05-13 00:44:06 -03:00
Adam Newbold	51afa7113a	Update ai_robots_update.yml with rebase command to fix scheduled run	2025-12-03 20:15:10 -05:00
Adam Newbold	2ccd443581	Update ai_robots_update.yml with workflow_dispatch ... Adding workflow_dispatch to enable manual triggers of this schedule job (for testing)	2025-12-01 20:24:41 -05:00
Cory Dransfeldt	899ce01c55	chore(ai_robots_update.yml): correct workflow by revising git flags + adding guard	2025-06-02 14:56:09 -07:00
Patrick Evans	498aa50760	lint robots.json during pull requests	2025-05-15 11:15:25 -05:00
Dennis Camera	8a74896333	Add workflow to run tests on pull request or push to main	2025-02-18 20:30:27 +01:00
Glyn Normington	b7f908e305	Merge pull request #66 from fabianegli/patch-1 ... Allow Action to succeed even if no changes were made	2025-01-07 03:54:40 +00:00
Cory Dransfeldt	8e98cc6049	Merge pull request #61 from glyn/improve-naming ... Rename Python code	2025-01-06 08:10:47 -08:00
Fabian Egli	30ee957011	bail when NO changes are staged	2025-01-06 12:05:42 +01:00
Fabian Egli	83cd546470	allow Action to succeed even if no changes were made ... Before, the Action would fail in case there were no changes made to any files by the converter.	2025-01-06 11:39:41 +01:00
Glyn Normington	c01a684036	Convert robots.json more frequently ... Specifically, when github workflows or code is changed as either of these can affect the conversion results. Ref: https://github.com/ai-robots-txt/ai.robots.txt/issues/60	2025-01-05 05:03:50 +00:00
Glyn Normington	9e372d0696	Ensure dependency installed ... Ref: https://github.com/ai-robots-txt/ai.robots.txt/issues/60#issuecomment-2571437913 Ref: https://stackoverflow.com/questions/11783875/importerror-no-module-named-bs4-beautifulsoup	2025-01-05 01:45:33 +00:00
Glyn Normington	996b9c678c	Improve job name ... The purpose of the job is to convert the JSON file to the other files.	2025-01-04 05:28:41 +00:00
Glyn Normington	3a43714908	Rename Python code ... The name dark_visitors.py gives the impression that the code is entirely related to the dark visitors website, whereas the update command relates to dark visitors and the convert command is unrelated to dark visitors.	2025-01-04 04:55:34 +00:00
fabianegli	b64284d684	restore correct attribution logic to before PR #55	2024-11-26 09:41:46 +01:00
Y. Meyer-Norwood	e8f0784a00	Explicitly use release tag for checkout	2024-11-13 10:26:37 +13:00
Y. Meyer-Norwood	94ceb3cffd	Add authentication for gh command	2024-11-11 13:04:55 +13:00
Y. Meyer-Norwood	adfd4af872	Create upload-robots-txt-file-to-release.yml	2024-11-11 12:58:40 +13:00
fabianegli	3ab22bc498	make conversions and updates separately triggerable	2024-10-19 19:56:41 +02:00
fabianegli	7e2b3ab037	rename action	2024-10-19 19:09:34 +02:00
fabianegli	0c05461f84	simplify repo and added some tests	2024-10-19 13:06:34 +02:00
Fabian Egli	25adc6b802	log git repository status	2024-10-19 00:28:41 +02:00
Fabian Egli	b584f613cd	add some signposts to the log	2024-10-19 00:13:09 +02:00
Fabian Egli	b3068a8d90	add some signposts	2024-10-19 00:12:25 +02:00
Fabian Egli	a46d06d436	log changes made by the action in main.yml	2024-10-19 00:04:15 +02:00
Fabian Egli	cfaade6e2f	log the diff in the update action daily_update.yml	2024-10-19 00:01:15 +02:00
nisbet-hubbard	cc18b8617c	Update main.yml	2024-09-03 07:48:48 +08:00
Chenghao Mou	6f96795edc	restore cron	2024-08-07 12:43:44 +01:00
Chenghao Mou	b00067bc86	restore files deleted by failed workflow and fix main commit message	2024-08-07 12:36:21 +01:00
Chenghao Mou	366e49dc6d	restore files deleted by failed workflow and fix main commit message	2024-08-07 12:21:40 +01:00
Chenghao Mou	09c6b78b46	fix job dependency	2024-08-07 11:45:37 +01:00
Chenghao Mou	944bee0f56	call main after update	2024-08-07 11:31:58 +01:00
Chenghao Mou	0b6eba8dd5	skip push if no change	2024-08-06 19:41:38 +01:00
Chenghao Mou	379c339f97	skip push if no change	2024-08-06 19:41:38 +01:00
Chenghao Mou	8c6482fb45	restore the cron	2024-08-06 18:12:41 +01:00
Chenghao Mou	55e92f4324	update existing ones	2024-08-06 17:48:06 +01:00
Chenghao Mou	52d54cf127	restore the cron	2024-08-06 17:28:07 +01:00
Chenghao Mou	8ab1e30a6c	test workflow	2024-08-06 17:12:26 +01:00
Chenghao Mou	192bf67631	add dark visitor workflow	2024-08-06 17:02:23 +01:00
John Bowdre	b0a93aeb70	only build on changes to robots.json	2024-08-04 17:45:18 -05:00
Adam Newbold	1fdc79dacb	Adding GitHub Action	2024-08-01 18:17:19 -04:00