Fern 0.1 posts 73.4 on SWE-Bench Verified, far behind frontier models on agentic benchmarks
bneyshabur · x · 2026-10-26
Fern 0.1 (Mirendil-Base plus a short post-training round) scored 44.2 on DeepSWE v1.1, 53.2 on SWE Atlas Codebase QnA, 76.5 on SWE-Bench Pro v2-Hard, and 73.4 on SWE-Bench Verified — competitive with open-weight peers like Beam but well behind Kimi-K3 (93.4 SWE-Bench Verified) and Claude Opus 5.5 (98.2; Terminal Bench 66.4 vs Fern's 6.1). The team positions Fern for high-volume agentic work — auditing training data, debugging runs from logs, reading agent traces end to end — rather than leaderboard chasing.
Related event: Fern Team Argues 'Context Is Intelligence'; Debuts at 73.4 on SWE-Bench(2 posts)→
More from coding & agent
- Agent verified 'success' after double-voting: a lesson in harness vs model bugs — TejasKumar_ · 2026-10-26
- diginomica: agentic automation hype is overdone, and 'The Hackening' of enterprise security has begun — jonerp · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Devin launches Security suite powered by Claude Mythos with $100M+ credits — DevinAI · 2026-10-26
- HackBench injects reward hacks into agent trajectories to stress-test monitors — DanielKhashabi · 2026-10-26
- Hugging Face ships hf CLI updates: deduped cache saves 19GB, faster buckets, retryable GPU jobs — yb2698 · 2026-10-26