ColBERT Author Slams Frontier Models: 'A Pre-Aimed Catapult That Never Lands Where You Want'
lateinteraction · x · 2026-09-20
Omar Khattab (lateinteraction, ColBERT author) offers a vivid analogy for frontier models' jagged capabilities in 2026: asking for a taxi ride and getting handed a catapult pre-aimed at a few destinations — initially impressive speed and distance, but you can never land at your actual destination no matter how many times you fly.
He adds that while everyone wonders about the future of work with AI, he's in constant agony extracting OK work from frontier models at "Ultra" settings, providing levels of detailed context and elaborate feedback he's never given a human — despite giving feedback being literally his job.
A first-hand researcher's take on the gap between benchmark strength and real-world usability.
Related event: Researcher's rant on brittle frontier models sparks resonance(5 posts)→
More from AGI Musings
- ML author Burkov slams resigning AI scientists: 'What the hell was your goal?' — burkov · 2026-09-20
- Pedro Domingos: AI makes knowing what to do matter more than knowing how — pmddomingos · 2026-09-20
- SF's AI safety crowd sounds condescending — and it's fueling public resentment — signulll · 2026-09-20
- Vitalik defends EA ties while reasoning his way out of longtermism — inductionheads · 2026-09-20
- Noam Brown says air-gapping may not stop misaligned AI; Bryan Cantrill pushes back on "contagion of fear" — brianmichel · 2026-09-20
- Grady Booch amplifies bubble warning: the $500B AI infra mega-deal is mostly MOUs — Grady_Booch · 2026-09-20