Context layers don't violate the bitter lesson: LLM-era indexes are data movement economics
CShorten30 · x · 2026-11-23
A technical debate on AI-era context/semantic layers, arguing from database history:
- DB history lesson: query processing has always been I/O or memory bound; improving performance means transferring less data to the LLM or faster — exactly what an index/context layer does
- Quality is the new challenge: B-trees don't affect correctness (same rows as a full scan), but a context layer is lossy by design — the LLM only sees what you put in context, possibly requiring progressive disclosure
- On the bitter lesson: @eddiedzhou argues a context layer is just systems infra — local read replicas + derived indexes built by offline LLMs (no hardcoded ontologies, recompute as models improve). Search is literally half of Sutton's point; CPUs getting orders of magnitude faster didn't kill B-trees or materialized views. Even at 0ms frontier inference, live fan-out across scattered SaaS APIs stays I/O-bound, and proprietary state is the last SaaS moat
Related event: Does the Context Layer Violate the Bitter Lesson? A Technical Debate(2 posts)→
More from AGI Musings
- ESR: it's already too late to suppress open-weight models — and that's a good thing — kingslyj · 2026-11-23
- Gary Marcus asks: what system actually powered Alpoge's Riemann hypothesis AI work? — GaryMarcus · 2026-11-23
- P=NP wouldn't break security — polynomial time isn't practical time — alex_peys · 2026-11-23
- Blogger argues non-frontier AI labs like xAI and Mistral should pivot to inference providers — DionysianAgent · 2026-11-23
- OpenAI's Jakub Pachocki: pretraining avalanche of advances conceivable by 2027 — scaling01 · 2026-11-23
- "Many think they're centaurs; actually they're just being dragged by the horse" — Pseudomanifold · 2026-11-23