DeepSeek-V4.1-Flash (552B) runs at 38.2 tps on a single AMD Ryzen AI Max+ 395 mini PC
solyarisoftware · x · 2026-11-06
A new open-source inference engine called libosmos runs the enormous DeepSeek-V4.1-Flash — 552B backbone parameters plus 196B Engram parameters, 16B active per token — on a single AMD Ryzen AI Max+ 395 mini PC with 128GB unified memory and Radeon 8060S iGPU, no multi-GPU server needed.
Developer-reported benchmarks with Unsloth's UD-IQ1S quantization on Linux + ROCm 7.x:
- 38.2 tps decode with speculative decoding (19.2 tps without)
- 482 tps on 8K prompt processing
- 1M configured context support
Compared against Project Maya 1.1.53 on the same hardware and quantization, libosmos edges it out: 38.2 vs 35.4 tps with drafting.
More from Infra
- USM turns idle NVIDIA VRAM into Linux swap, ~9 µs vs 55 µs on NVMe — kha333n · 2026-11-06
- Opening a KTV next to Ulanqab's AI data centers is the new 'sure-fire' business, jokes Chinese AI circle — basedjensen · 2026-11-06
- Musk claims SoftBank is defaulting on its turbine orders for AI datacenters — SumitGup · 2026-11-06
- Thinking Machines' Inkling 2 Small trained on 12.55T tokens, with RL consuming over a quarter of pretraining compute — YouJiacheng · 2026-11-06
- SpaceX plans ~6 GW Arizona datacenter across 9 buildings, no industrial water — elonmusk · 2026-11-06
- 7 techniques for building cost-efficient AI pipelines, from model routing to cost-per-task — goyalshaliniuk · 2026-11-06