Local 35B authored 800 math problems under cloud-agent supervision — authoring erred 5x more than cold-solving
Individual_Page9676 · reddit · 2026-10-25
A Redditor ran a full self-authored training-and-grading pipeline: a cloud frontier-model agent (connected to a local Mac M5 Max setup via Tailscale + LM Studio) supervised a local Qwen3.6-35B-A3B (375-400 tok/s) that wrote 800 original math problems, then cold-solved each blind in fresh sessions, with disagreements escalated to a 122B-A10B judge and finally brute-forced in Python by the agent.
Key finding: cold-solving is far more reliable than authoring official solutions — in adjudicated disputes the cold solver was right 56 times vs. the author's 12 (roughly 5:1). "Authoring is the error-prone mode. Cold-solving is the filter."
Results across three rounds: 731/800 problems verified clean, 62 had wrong authored keys (kept with corrected answers), 7 excluded as flawed; final validation re-solved all 793 keys with 774 confirmed outright. Over 4.1M completion tokens consumed — all local, zero API spend.
Notable catches include a problem whose flawed key came from the agent itself, and 4 problems that turned out to be genuinely flawed (multiple valid answers, mutually inconsistent conditions). The whole author-blind-solve-verify-adjudicate pipeline is a replicable template for building verified synthetic datasets cheaply.
More from coding & agent
- Code is clay now: an iOS developer on how agentic coding reshapes software building — bibryam · 2026-10-25
- Row-Bot 5.3 ships embedded browser, tiered approvals and 40% lighter context for local-first agent — Acceptable-Object390 · 2026-10-25
- Task-specific inference engines emerging as specialization costs collapse, argue researchers — _AustinCalvert_ · 2026-10-25
- Daniel Miessler on models vs harnesses: better models mean less harness — _AustinCalvert_ · 2026-10-25
- Celesto: open-source secure persistent computer for AI agents — aniketmaurya · 2026-10-25
- Manning offers free 4-chapter ebook on building testable, reliable AI agents — kalyan_kpl · 2026-10-25