Agent fine-tunes Qwen3.5-0.8B with synthetic data: 4.7%→76.3% accuracy, 11.9× faster
ben_burtenshaw · x · 2026-10-16
A demo from Hugging Face's benburtenshaw shows the Beam agent fine-tuning a decision model for a codebase: it built a synthetic incident-triage dataset and fine-tuned Qwen3.5-0.8B with Unsloth.
Results:
- Exact-match accuracy: 4.7% (prompted zero-shot) → 76.3% (fine-tuned)
- Latency per ticket: 1,218ms → 103ms, an 11.9× speedup
A related demo ran Beam on ProgramBench (rebuilding a program from its binary) under the same token budget across three orchestration modes: solo, 5 peers sharing a git repo, and lead + subagents.
It's a replicable "big model generates data, small model gets fine-tuned" workflow.
More from coding & agent
- Deep Research API showdown: OpenAI, Perplexity, Parallel, Tavily, Gemini, Valyu compared — bar_raiser333 · 2026-10-16
- Geoffrey Huntley open-sources Overalls, a self-hosted AI developer platform under MIT — kieranklaassen · 2026-10-16
- Dev shows 199.7bn tokens over 200 days on OpenAI Codex, peak 8.6bn/day — Dimillian · 2026-10-16
- Ask Claude to open its changes in nvim and scroll to the exact line for review — max__drake · 2026-10-16
- Proof CEO on x401: KYC-compatible agent identity using NIST IAL2 verifiable credentials, composing with x402 payments — csuwildcat · 2026-10-16
- Pydantic's HTTPX2 adds native WebSocket and SSE support — samuelcolvin · 2026-10-16