Exa says it now serves 80B pages and tracks 1.4T URLs
yoimnotkesku · x · 2026-08-04
Exa says it now serves 80B pages and tracks 1.4T URLs, with a path to Google-scale indexing in early 2027.
- It estimates web indexes roughly as: Google 1T pages, Bing 500B, Yandex 200B, Brave 40B.
- The company says most of the web is low-quality noise, so crawlers must fetch far more than they serve and train filters to remove junk.
- Exa also claims peak QPS matters: Bing is estimated around 30k/s, Google around 500k/s.
- Because agent traffic often fans out into many sub-searches and can spike during RL workloads, Exa says it has to design for highly bursty usage.
More from Companies & People
- Cloudflare’s agents package tops 1.29 million weekly npm downloads — whoiskatrin · 2026-08-04
- UNESCO and LG AI Research launch a free 11-language AI ethics MOOC — moniquejmorrow · 2026-08-04
- Visa to buy AI fraud firm BioCatch for $2.4B in cash — Polymarket · 2026-08-04
- Superblocks pitches enterprise vibe coding that keeps company data inside AWS — Scobleizer · 2026-08-04
- Bittensor subnet 107 says OpenAI co-authored a field report on agentic scientific computing — markjeffrey · 2026-08-04
- A startup meme skewers founders who keep asking for a technical co-founder — wavefnx · 2026-08-04