DeepSeek-based agent harness tops Lean Eval at a fraction of the cost, talk reveals
lukaszkaiser · x · 2026-10-23
- Alex Skidanov's talk at SF Lean Meetup presents a DeepSeek-based agent harness that topped the prestigious Lean Eval benchmark at a fraction of the cost of frontier models.
- The talk also covers how much you can actually trust Lean proofs produced by models, and the techniques the team uses to improve that trust (segment starting at 30:00).
- A notable signal for formal math + LLMs: a cheap base model plus a well-engineered harness can beat much pricier setups.
More from coding & agent
- Codex Dot's unlimited-usage delegator design avoids quota starvation in agent orchestration — ChrisRackauckas · 2026-10-23
- 7 Mistakes Economists Make With Claude Skills: Not Building Their Own and Making Too Few — aniketapanjwani · 2026-10-23
- MIT Sloan research: AI speeds up coding but delivery still bottlenecks on testing — sven_ai · 2026-10-23
- Agents logging into bank accounts violate most banks' terms, even with user consent — pstAsiatech · 2026-10-23
- Codex desktop shipped with no editor: 4-person team bet everything on agents — sujingshen · 2026-10-23
- Under matched token budgets, most gains of web agent online augmentation vanish — hllo_wrld · 2026-10-23