Cloudflare — Internet-scale bot-vs-human telemetry: per-domain, per-crawler-class request data distinguishing AI-training crawlers from search crawlers and human visitors, with crawler identity (GPTBot, ClaudeBot, Bytespider, PerplexityBot, Googlebot), declared crawl purpose (training / search / aggregation), crawl volume, and the matched referral traffic back to the crawled property — yielding a crawl-to-click ratio per AI platform per publisher. Aggregated in Radar AI Insights; underlying form is edge request telemetry across millions of customer domains.. Very large: ~5,200 employees, ~$2.…
Cloudflare — Internet-scale bot-vs-human telemetry: per-domain, per-crawler-class request data distinguishing AI-training crawlers from search crawlers and human visitors, with crawler identity (GPTBot, ClaudeBot, Bytespider, PerplexityBot, Googlebot), declared crawl purpose (training / search / aggregation), crawl volume, and the matched referral traffic back to the crawled property — yielding a crawl-to-click ratio per AI platform per publisher. Aggregated in Radar AI Insights; underlying form is edge request telemetry across millions of customer domains.. Very large: ~5,200 employees, ~$2.…
Cloudflare offers (Alternative, Sentiment, Geospatial) — Enormous: bot/human classification events in the 10^12+/month range raw, since a large share of global HTTP traffic passes the edge. The realistically licensable slice — per-domain daily crawl-purpose counts plus matched referral counts — is still billions of aggregate rows per year across millions of domains.
Bot/classifier training (bot vs human, crawler-identity identification); measuring AI-engine content appropriation per publisher — a direct input to publisher ad-revenue and subscription-churn models and to pay-per-crawl pricing; building content-licensing comparables for AI training-data negotiations; web-redundancy and crawl-efficiency studies; training agents to act on the open web; crawler-share shifts as a leading indicator of AI-model release cadence
The data is with 5 years of history, delivered by api.
Coverage spans US; IT Services; alternative, sentiment, geospatial; equities.
The supplier's own access route is an API key or free sign-up. Request a sample through Brickroad to start.
Not yet on file: Summary & mission · Use cases & buyers · FAQ