METR CEO: AI models may match human researchers by August 2027
theamberyang · x · 2026-09-15
METR CEO Rayan Krishnan joined Bloomberg to discuss AI progress and independent evaluation:
- We're better at building AI than understanding it; scaling testing matters more than slowing down
- METR's RSI Index projects models could match human researchers on tested tasks by August 2027; embedded evaluators can give more accurate estimates from internal systems
- Public conflict masks cooperation—labs, policymakers and enterprises all want better evidence
- Independence has costs: METR has rejected contracts that would compromise it, arguing the tester shouldn't also sell the solution
- Evaluation should scale through better technology, not become a bureaucratic moat; market-based evaluation has a role
More from AGI Musings
- Ethan Mollick on AI and work: 'You're a hand-weaver in a world of power looms' — emollick · 2026-10-30
- Vercel founder Rauch: trust is the deepest moat for a post-AGI business — soleio · 2026-10-30
- Scattered Across AI Assistants: A New Kind of Cognitive Failure Nobody Has a Prior For — manosaie · 2026-10-30
- a16z partners: 10-20M people drive $150-200B in token spend, and AI workspaces will eat the browser — petergyang · 2026-10-30
- Professor Who Turned Down OpenAI in 2018 Reflects on PhD Anxiety in the AI Era — SharonYixuanLi · 2026-10-30
- Plinz: Values must be discovered, not given — they emerge at the superorganism level — twiecki · 2026-10-30