Engineer uses Claude Code to build tiny CPU neural nets hitting ~6k tok/s per core
GregoryDiamos · x · 2026-09-07
Engineer Gregory Diamos tasked Claude Code with exploring 'outrageously small neural nets' for CPU inference and got three interesting discoveries, reaching 6k tok/s per core. He cautions it's experimental—production use would need a multi-trillion-token foundation run plus fine-tuning, taking months on a 32-core CPU.
More from coding & agent
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- Stop building agents that can't change models: stable contracts, external state, real evals — rseroter · 2026-10-27
- Cheap models can cost more in agent harnesses than frontier ones, researcher warns — vasuman · 2026-10-27
- Kanzo: A 5k-Line Rust Kanban Binary With MCP for Claude Code and Codex — allmudi · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27