Liquid AI hits 21k+ output tokens/s on NVIDIA B300 at high concurrency
JosephJacks_ · x · 2026-10-14
Liquid AI demonstrated its inference stack scaling well on GPUs: over 21,000 output tokens/s on an NVIDIA B300 at high concurrency. The post highlights the same models running blazingly fast from a MacBook all the way to Blackwell GPUs.
More from Infra
- Same model, 9x cost gap: Artificial Analysis publishes cache hit rates per provider — ArtificialAnlys · 2026-10-14
- Apple's Minions turns idle Mac GPUs into a pooled cloud for ML inference — MikePFrank · 2026-10-14
- AnythingLLM ditches Ollama for llama.cpp, with up to 2x faster inference — solyarisoftware · 2026-10-14
- LlamaForge: a browser panel for upstream llama.cpp with local agents that must quote their sources — Sleepybear2611 · 2026-10-14
- CoreWeave's GPU fleet is mostly older boxes, raising questions about depreciation math — ivan_bezdomny · 2026-10-14
- NVMe reads the 7GB checkpoint in 2s, so why does ComfyUI take 30s to load? — rguerraf · 2026-10-14