12,000 tok/s inference achieved on a $250 FPGA board
ycombinator · x · 2026-08-17
Lamb Labs demonstrated a proof-of-concept achieving 12,000 tokens per second inference on a $250 AMD KV260 FPGA board. The project, named "little lamb," runs a small model entirely within FPGA fabric with resident weights in on-chip memory, eliminating DRAM from the token loop. The authors note that while the coherence is poor and it is not intended for good chat results, the extreme speed showcases the potential for specialized hardware acceleration.
Related event: $250 FPGA board hits 12,000 tokens/s inference(3 posts)→
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16