Dev's CPU-native LLM Architecture Hits 113-130 tok/s on a 10B Model, Quality Lags
WildPino25 · reddit · 2026-09-15
Reddit user WildPino25 presents SiliconLLM, a CPU-native LLM architecture that runs a 10B-parameter model at 113-130 tok/s on a Ryzen 5 3600X with no GPU — though the weights are low quality. To test quality at scale, he started training a 206M model on a T4 (8+ weeks) and tried converting Qwen2.5-Coder to his SSM/ternary/sparse format, finding donor adaptation painfully hard. The repo and research branch are on GitHub.
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02