From fe556254f1b789d57af081eac310a20da8b16bbf Mon Sep 17 00:00:00 2001 From: brib Date: Thu, 12 Mar 2026 11:29:37 +0000 Subject: [PATCH 1/2] Add some more studies on LLMs and code quality A few weeks ago I researched the impact of LLMs and code quality to write a section for this list. This PR incorporates this research into the existing writeup. --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 75f6995..b0a1a49 100644 --- a/README.md +++ b/README.md @@ -335,6 +335,7 @@ Vibe coding / agentic workflows result in poorer code quality, and relaxed overs * [How I Dropped Our Production Database and Now Pay 10% More for AWS](http://archive.today/2026.03.06-144058/https://alexeyondata.substack.com/p/how-i-dropped-our-production-database) * [Claude Tested Everything Except the One Thing That Mattered (Ai agent refuses to follow explicit instructions to test `createPost()` in increasingly erratic ways)](http://archive.today/2026.03.09-201135/https://christophermeiklejohn.com/ai/claude/2026/03/08/claude-tested-everything-except-the-one-thing-that-mattered.html) * [Amazon calls engineers for a “deep dive” internal meeting to discuss “GenAI”-related outages](https://ghostarchive.org/archive/3TfgF) +* GitClear has released reports in [2024](https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality) and [2025](https://www.gitclear.com/ai_assistant_code_quality_2025_research) indicating a worsening of key code quality metrics correlating with increased LLM adoption. ### Infosec risks @@ -342,8 +343,10 @@ This also often results in massive security holes. * [Meta Security Researcher's AI Agent Accidentally Deleted Her Emails](http://archive.today/2026.02.26-153034/https://www.pcmag.com/news/meta-security-researchers-openclaw-ai-agent-accidentally-deleted-her-emails) * [Moltbook’s ‘vibe-coded’ breach is the future of security failures](https://thehill.com/opinion/cybersecurity/5744310-ai-powered-security-risks/) +* [In a study evalulating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code](https://arxiv.org/abs/2508.21634) +* [LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks](https://www.theregister.com/2025/04/12/ai_code_suggestions_sabotage_supply_chain/) ([Arxiv study](https://arxiv.org/abs/2406.10279)) -## Healthy and Safety +## Health and Safety There's been a number of high profile incidents that have resulted in endangerment or death. Here's some examples: From e41b36ed8c7d04fe9715a2e27077a905036be231 Mon Sep 17 00:00:00 2001 From: brib Date: Thu, 12 Mar 2026 11:36:24 +0000 Subject: [PATCH 2/2] Undo typo change --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b0a1a49..905187d 100644 --- a/README.md +++ b/README.md @@ -346,7 +346,7 @@ This also often results in massive security holes. * [In a study evalulating over 500k code samples, LLM-generated code was found to contain more high-risk security vulnerabilities than human-generated code](https://arxiv.org/abs/2508.21634) * [LLMs make up package names, making them vulnerable to incorporating malicious code in "slopsquatting" attacks](https://www.theregister.com/2025/04/12/ai_code_suggestions_sabotage_supply_chain/) ([Arxiv study](https://arxiv.org/abs/2406.10279)) -## Health and Safety +## Healthy and Safety There's been a number of high profile incidents that have resulted in endangerment or death. Here's some examples: