remote/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2026-09-02 22:34:03 +02:00
Code Issues Projects Releases Packages Wiki Activity

test: pin the SLCC1/SLCC2 false positive from #208

Browse source 
The non-AI user agent list covers synthetic prefix/suffix variants
(NotCursor, CursorNot). It had no real-world agent that embeds a listed
name mid-string, which is what #208 actually reported: SLCC1 and SLCC2 in
older Internet Explorer and Trident agents span the listed agent LCC.

Verified the case is load-bearing: removing the word boundaries from
list_to_pcre fails this test with
  AssertionError: <re.Match object; span=(65, 68), match='LCC'> is not None

Also document in the FAQ that agent names are matched as whole words, for
anyone consuming robots.json directly and writing their own matcher.
This commit is contained in:
Saliu Jamiu Olamilekan 2026-08-18 20:28:19 -07:00
commit bcc4e408e2
2 changed files with 22 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

7
code/tests.py
View file

@ -126,6 +126,13 @@ class TestUserAgentPatternGeneration(unittest.TestCase):
"NotAmazonbot/1.0",
"NotApplebot/1.0",
"NotBytespider/1.0",
# Real-world user agents that embed a listed bot name mid-string.
# These older Internet Explorer / Trident agents contain "SLCC1" or
# "SLCC2" (a Windows licensing component), which spans the listed
# agent "LCC". Reported in #208 and fixed by the word boundaries
# added in #260; pinned here so the specific report cannot regress.
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.30729)",
"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; Media Center PC 6.0)",
]
for ua in non_ai_user_agents: