Tim Shi on AI Autonomous Research: Verification Is the Bottleneck as Task Length Doubles
AGI House 发布与 Tim Shi(早期 OpenAI 成员、Cresta 与 Recursive 联合创始人)的对谈纪要,核心议题是 AI 能否自主做科研——提出想法、做实验、决定下一步——以及我们离自我改进的 AI 还有多远。
已确认
- Tim Shi 早年与 Andrej Karpathy 在 OpenAI 做过「用键盘鼠标操作浏览器的 agent」,比 computer use 产品化早近十年,当时因缺少基础模型而失败(m3、m5)
- 谈及 Cresta 经历时,他表示团队每两年重写技术栈,认为规模胜过修补(m3)
- 他援引 METR 数据:AI 能独立完成的任务长度每几个月翻倍,当前处于指数增长段;但算力、能源等仍是约束(m2)
- Recursive 打造了 Eureka Machine,让 AI 自主运行「提出想法-实现-评估-再选下一个想法」的科研闭环,产物之一是快速本地编码模型 Flint,据称在相关基准上表现突出(m4)
- 关于安全,他关注 AI 辅助黑客攻击等近期风险,主张能力很强的模型需要 know-your-user 身份核验流程,并认为对齐研究本身也可以被自动化(m7、m8)
- 访谈线程还指出 AI 自主科研的瓶颈在验证环节:需要大规模训练的 evals、刷分式 reward hacking,以及生物领域的湿实验与临床试验(m1)
为什么重要
- 该访谈系统梳理了 AI 自主科研从想法生成到验证的完整链路,指出 AI 产出想法的速度加快后,验证基础设施成为新的关键制约
- Eureka Machine 提供了一个已在运行的全自动科研循环实例,而 METR 的任务长度翻倍数据为判断自我改进 AI 的到来时间提供了量化参考
- know-your-user 与对齐研究自动化的主张,为强模型时代的治理提供了来自一线创业者的具体方案
2026-11-21 ~ 2026-11-21 · 8 related posts
Primary sources
- Early OpenAI's Tim Shi on how close we are to self-improving AI — agihouse_org ·
- Validation is the bottleneck: evals, wet labs and trials slow AI-driven research — agihouse_org ·
- Recursive's Eureka Machine automates the full AI research loop — agihouse_org ·
- Early OpenAI's Tim Shi built keyboard-mouse web agents with Karpathy a decade before computer use — agihouse_org · 2026-11-21
- [source] Early OpenAI's Tim Shi on how close we are to self-improving AI — agihouse_org · 2026-11-21
- Cresta rebuilt its stack every 2 years: scaling beats patching, says Tim Shi — agihouse_org · 2026-11-21
- [source] Recursive's Eureka Machine automates the full AI research loop — agihouse_org · 2026-11-21
- [source] Validation is the bottleneck: evals, wet labs and trials slow AI-driven research — agihouse_org · 2026-11-21
- METR data: AI task-completion length doubles every few months, says Tim Shi — agihouse_org · 2026-11-21
- Tim Shi: capable AI models need know-your-user processes against hacking — agihouse_org · 2026-11-21
- Tim Shi: very capable models need a know-your-user process; alignment research can be automated — agihouse_org · 2026-11-21