Taalas burns LLM into silicon: 8B model hits 17,000 tok/s on a single chip, 213x an RTX 5070

IAM_274 · reddit · 2026-10-21

Taalas's ChatJimmy runs an 8B Llama model at 17,000 tok/s on a single chip by literally burning the model into silicon — the LLM is the chip, with all compute going straight to the model's matrix math. An RTX 5070 manages 80 tok/s on the same model, 213x slower.
It's an experimental proof of concept, not a product. The poster speculates 27B models at 400k tok/s, or buying your favorite model's chip for the price of an iPhone.
Key tradeoff: models can't be swapped; knowledge cutoff means buying a new chip every 3-5 years as the burnt-in model ages.
Manufacturing cost and power consumption reportedly make classic GPUs look embarrassingly expensive, per a linked Medium deep-dive.

More from Infra