remote/ai.robots.txt
1
0
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2026-09-28 21:04:16 +02:00
Code Issues Projects Releases Packages Wiki Activity

add Lightpanda user agent

Browse source 
The Lightpanda browser is an AI-centric headless browser used for scraping and automation.

It is currently being used by a botnet of millions of IP addresses, including over 20k SpaceX IPs, to scrape sites such as the official Haskell Gitlab:
https://github.com/lightpanda-io/browser/issues/3156
This commit is contained in:
Anthony M. Cook 2026-08-07 11:22:16 -05:00 • committed by GitHub
commit a5d17db8b8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 8 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

9
robots.json
View file

@ -665,6 +665,13 @@
"frequency": "Unclear at this time.",
"description": "Description unavailable from knownagents.com More info can be found at https://knownagents.com/agents/lcc"
},
"Lightpanda": {
"operator": "Anyone who downloads the Lightpanda client. Possibly being used by a [Grok-adjacent](https://github.com/lightpanda-io/browser/issues/3156#issuecomment-5217843616) organization's botnet.",
"respect": "At the [discretion](https://github.com/lightpanda-io/browser/blob/b04c99a9111564ebe06317f644680eda5e3ee83e/src/help.zon#L385) of Lightpanda users.",
"function": "AI Data Scrapers",
"frequency": "Defined per-user.",
"description": "Lightpanda is a custom-built headless browser designed for AI and automation."
},
"LinerBot": {
"operator": "Unclear at this time.",
"respect": "Unclear at this time.",
@ -1141,4 +1148,4 @@
"frequency": "Unclear at this time.",
"description": "Description unavailable from knownagents.com More info can be found at https://knownagents.com/agents/zanistabot"
}
}
}