LFM2.5-24B-A2B hits ~300 tok/s on a single DGX Spark via TandemLLM
JosephJacks_ · x · 2026-10-15
Liquid AI's LFM2.5-24B-A2B now runs on TandemLLM, installable in one click on a DGX Spark and reaching 300 tok/s for chat.
The author also produced EXL3 quants from 2.5 to 6.0 bpw: the default 3.5 bpw fits in 11.2 GB while decoding at 151 tok/s, matching BF16's top token 86.7% of the time. On edits and copies, a lookup drafter pushes it to 269–414 tok/s with identical output.
More from Infra
- CoreWeave executives unload shares for months as stock slides, insider filings show — SumitGup · 2026-10-15
- Musk amplifies a16z compute-scarcity chart: old GPUs becoming appreciating assets — elonmusk · 2026-10-15
- SK Hynix seeks US approval to ship chipmaking equipment to China plants as US factory condition — teortaxesTex · 2026-10-15
- CoreWeave co-founder courts insurers' 'unlimited' capital as private credit hits 35% of US insurer assets — SumitGup · 2026-10-15
- Go 1.27 makes HTTP/1 connection reuse ~5x faster: 100 requests drop from 710ms to 150ms — rakyll · 2026-10-15
- Open textbook "Model-Driven DevOps for Big Data Applications" now freely available — burkov · 2026-10-15