ended8월 8일· 1 sources
I published an open dataset of every AI crawler user agent (GPTBot, ClaudeBot + 26 more)
Why it matters
While building AI-crawler tooling I kept needing the same thing: a current, machine-readable list of AI crawler user agents. Every blog post has a partial list, half of them outdated, none of them str...
1
Sources
+0
24h
—
Growth
5d
Active