ColBERT Author Slams Frontier Models: 'A Pre-Aimed Catapult That Never Lands Where You Want'
lateinteraction · x · 2026-09-20
Omar Khattab (lateinteraction, ColBERT author) offers a vivid analogy for frontier models' jagged capabilities in 2026: asking for a taxi ride and getting handed a catapult pre-aimed at a few destinations — initially impressive speed and distance, but you can never land at your actual destination no matter how many times you fly.
He adds that while everyone wonders about the future of work with AI, he's in constant agony extracting OK work from frontier models at "Ultra" settings, providing levels of detailed context and elaborate feedback he's never given a human — despite giving feedback being literally his job.
A first-hand researcher's take on the gap between benchmark strength and real-world usability.
More from AGI Musings
- Andreessen: Agents Are Entering Enterprises as Chaos, Just Like PCs and Spreadsheets Did — Dan_Jeffries1 · 2026-11-04
- 1000+ mathematicians sign pledge to shun AI companies and models — Shesawthat · 2026-11-04
- Von Neumann meme: "You don't have to be responsible for the world that you're in" — akbirthko · 2026-11-04
- Joscha Bach on the dream chocolate that stopped being real — Machine Learning Street Talk · 2026-11-04
- François Chollet: modeling brain wiring as an engineering problem constrained by genome size and time — CatAstro_Piyush · 2026-11-04
- Most US Couples Now Meet in a Data Centre, Doubling Office Matches — menhguin · 2026-11-04