17M-parameter model beats frontier LLMs 80% of the time after cheap synthetic-data finetuning
max_paperclips · x · 2026-09-27
MaximeRivest shares finetuning experiments: a 17M-parameter Ettin model finetuned on LLM-generated synthetic data beats Jev 80% of the time, and beats Kimi-k3 40% of the time on human labels. Distilling from Kimi-k3 synthetic data gets 70% close to Jev.
Key numbers:
- Synthetic data generation: 8 seconds to 7 minutes, costing $0.77–$30
- Finetuning takes 45s–3 min on a 3090 GPU, 10–40 min on CPU, and even 20 min–3 hrs on a Samsung S21 phone GPU
- General models may still win when training data doesn't cover the test/prod distribution
Takeaway: with 500k+ inputs to classify, finetuning a small model on synthetic data is both faster and cheaper than general LLMs.
More from Models
- $200 Claude sub with Opus 5.5 feels like 5x the usage of $200 Codex with Astra — sethlazar · 2026-09-27
- Opus 5.5 demos nail spatial perception, but tactile sensing remains VLA robotics' frontier — bookwormengr · 2026-09-27
- Dev argues Gemini has achieved nothing like Google's 'google it' cultural moment — signulll · 2026-09-27
- Grok launches Finance integration to manage your bank and investment accounts — NicoVerderosa · 2026-09-27
- Factory AI CEO: 90% of tokens will go to open models within 12 months — matanSF · 2026-09-27
- banteg: astra finds nothing to flag after Opus 5.5, unlike nitpicky prior models — banteg · 2026-09-27