Brickroad
Brickroad
IndexSignalsBlog
/Index//

Product

  • Information Frontier Agent
  • Vendor Management Agent
  • Product tour
  • Pricing

Solutions

  • Utility estimation
  • Context assembly
  • Market intelligence

Suppliers

  • Sell your data
  • Start onboarding
  • Programmatic Data License

Community

  • Public index
  • Signals
  • Blog

Company

  • About
  • Team
  • Careers
  • Brand

Connect

  • LinkedIn
  • X
PrivacyTermsCookies

© 2026 Brickroad

/Index/Cloudflare/Internet-scale bot-vs-human telemetry
C

Internet-scale bot-vs-human telemetry

CLOUDFLAREA3 · direct accessManage supplier listing

Internet-scale bot-vs-human telemetry: per-domain, per-crawler-class request data distinguishing AI-training crawlers from search crawlers and human visitors, with crawler identity (GPTBot, ClaudeBot, Bytespider, PerplexityBot, Googlebot), declared crawl purpose (training / search / aggregation), crawl volume, and the matched referral traffic back to the crawled property — yielding a crawl-to-click ratio per AI platform per publisher. Aggregated in Radar AI Insights; underlying form is edge request telemetry across millions of customer domains.

Open buyer room →
Cloudflare/Internet-scale bot-vs-human telemetry
SampleCoverage

Sample

Brickroad customers

Sample this dataset before you buy it.

Your sourcing agent asks Cloudflare and files the sample in your catalog — private to you.

Start 7-day trial→Nothing is charged until the trial ends.

Coverage

Universe, instruments and categories.

Industry
IT Services
Instruments
equities
Categories
AlternativeSentimentGeospatial
Regions
US
Sample tickers
NETGOOGLTTDNWSANEWSDLBCRWV
Dataset card
Type
not stated
Format
Tabular time series (timestamped hourly/daily aggregates by domain, country, ASN, crawler user-agent, purpose) queryable through the Radar API; underlying form is structured HTTP edge request records
Volume
Enormous: bot/human classification events in the 10^12+/month range raw, since a large share of global HTTP traffic passes the edge. The realistically licensable slice — per-domain daily crawl-purpose counts plus matched referral counts — is still billions of aggregate rows per year across millions of domains
Users
~millions of Internet properties behind the network, of which several hundred thousand are paying customers (bounded from Cloudflare's public disclosures of 'millions of Internet properties'); Radar itself is additionally free-registered
History
5 years
Update frequency
not stated
Growth
Active — Radar AI Insights launched alongside this 2025 research line, crawl volumes rose 32% YoY in April before moderating to 4% in July, and pay-per-crawl has been productised
Launched
~2020 for Radar as a product (public beta 2020); the AI-crawl purpose-classification slice specifically launched ~mid-2025 with Radar AI Insights and pay-per-crawl
Delivery
API
Entity mapping
not stated
Sample
not stated
Point-in-time
not stated
Licence
not stated

supplier index read from their site

Cloudflare has not added their own details yet. Not yet on file:

  • Dataset
  • Data dictionary
  • Coverage
  • Provenance
  • Rights & data handling