Harness Doubles Down on Multi-Model Benchmark Scores
ycombinator · x · 2026-07-16
Poetiq claims their harness led to massive score improvements for several models on benchmarks.
- Reports indicate that Opus 4.8 / GPT-5.5 running on this system comprehensively outperformed Fable / GPT 5.6 Sol.
- Poetiq stated that their RSI system makes benchmarks "too easy." Given a benchmark, it builds its own solutions and beats SOTA across 6 categories: math, coding, planning, long context, tool usage, and web applications.
- They emphasized that no manual tuning was involved.
More from coding & agent
- Agent verified 'success' after double-voting: a lesson in harness vs model bugs — TejasKumar_ · 2026-10-26
- diginomica: agentic automation hype is overdone, and 'The Hackening' of enterprise security has begun — jonerp · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Devin launches Security suite powered by Claude Mythos with $100M+ credits — DevinAI · 2026-10-26
- HackBench injects reward hacks into agent trajectories to stress-test monitors — DanielKhashabi · 2026-10-26
- Hugging Face ships hf CLI updates: deduped cache saves 19GB, faster buckets, retryable GPU jobs — yb2698 · 2026-10-26