Dev trains a 50MB model in 15 min to replace Gemini Flash at 0.06s latency
newz2000 · reddit · 2026-09-26
A developer replaced a Gemini Flash boolean-classification call (99% accurate, 0.9s latency) with a small model he trained himself. Starting from 550 real examples expanded to 2,500 using two frontier models, training on an RTX A6000 took just 15 minutes. The resulting 50MB model runs on CPU in 0.06 seconds at 97% accuracy — acceptable for his use case. He plans server-side deployment with accuracy logging and is considering browser-side inference.
More from coding & agent
- Zero-dependency LLM SDK lm15 1.0 hits PyPI, supports 10+ providers and already used by dspy — lateinteraction · 2026-09-26
- Claude Code now wraps up tasks gracefully when hitting the 5-hour limit instead of cutting off mid-edit — lydiahallie · 2026-09-26
- Dev builds Mac voice-to-computer-use app with no LLM, using native speech-to-text plus JEV — niloofar_mire · 2026-09-26
- PR Lens turns every pull request into animated architecture diagrams, powering 2,500 PRs a month — nevrekaraishwa2 · 2026-09-26
- LangChain's founder: Jev + LangGraph is the combo for modeling agents as complex systems — hwchase17 · 2026-09-26
- Want Opus as main agent in Codex desktop? One dev is stuck between toolchains — kevinkern · 2026-09-26