Hobbyist's 348M model hits 99.4% on GPT-3 arithmetic tasks, beats 175B giant
nkthebass · reddit · 2026-09-10
A Reddit user trained a 348M-parameter model from scratch on 22.7B tokens to do arithmetic by showing its work — column addition with carries, borrow chains, partial-product multiplication. It averages 99.4% across the nine GPT-3 arithmetic sub-tasks, far above GPT-3 175B's direct-answer scores (e.g. 100% vs 9.3% on 5-digit addition).
The most interesting finding: the ceiling at 8 digits wasn't arithmetic but vocabulary — the model only had names for six place values, so at 9 digits it silently skipped a column and returned an answer one digit short, with every computed column perfect. Extending the place-name list from 6 to 19 entries raised the clean ceiling from 8 to 14 digits.
Other results:
- 3×3 multiplication 98%, negative results 85% (the weak step is magnitude comparison, not arithmetic)
- Reasoning traces are load-bearing: 95.3% valid-working-and-correct, only 0.7% valid-working-wrong-answer
- Weak spot is word problems: GSM8K 4%, ASDiv 16.5% — failure mode is operation selection, not arithmetic
More from Models
- Pixel analysis of OpenAI's Navier-Stokes chart suggests ~91 solved open problems kept unreleased — teortaxesTex · 2026-09-10
- Developer who spent €20k+ on Claude says models are degrading, urges skipping yearly plans — maier_ak · 2026-09-10
- User who spent €20k+ on Claude licenses says models keep getting weaker — maier_ak · 2026-09-10
- Users report AI assistant Astra omitting spaces, most often around numbers — gandamu_ml · 2026-09-10
- DeepSeek V4.1 flash reportedly has hidden vision support; official release still missing — teortaxesTex · 2026-09-10
- GPT-6 Astra vs GPT-5.6 Sol: Code Review Benchmark on 50 Real PRs — entelligenceai17 · 2026-09-10