TensorFold 1.2.5 ships local inference engine hitting 874 tok/s on Nemotron 3.5
HankYeomans · x · 2026-11-09
Open-source local inference engine TensorFold released v1.2.5, focused on running open models on your own GPUs with native Metal and CUDA support.
Key updates:
- New TensorFold Web local browser app for chat, models, learning and benchmarks
- CUDA adds LFM2.5-24B-A2B; Nemotron Parse and LightOnOCR-3 now on HIP
- Qwen3.8-27B MLX 8-bit runs 18% faster on cold prompts and 7.6% faster decode on M5 Ultra
- Lower memory use for Qwen CUDA streams, better conversation-cache handling, macOS PDF crash fix
Reported measurements: 874 tok/s decode for Nemotron 3.5 Lightning on RTX PRO 6000 Blackwell, and 199 tok/s aggregate across eight concurrent GLM-5.3-Flash streams on two M5 Ultras.
More from Infra
- Nvidia is the second-largest position in SPY, yet the largest position inside a SPY — dr1337 · 2026-11-09
- OpenAI CFO says datacenter expansion won't slow, stressing local community ties — i_dg23 · 2026-11-09
- 92T tokens over 1.54M DAU: the agent-era math feud over token burn — teortaxesTex · 2026-11-09
- Gary Marcus backs Ed Zitron: US data center capacity additions halved in Q4 2025 — GaryMarcus · 2026-11-09
- Analysts: Nvidia could stabilize near $155 if Huang is cleared, but an exit over security violations would hurt far more — GaryMarcus · 2026-11-09
- Texas gubernatorial candidate vows moratorium on all data center approvals — Traditional-Chip8339 · 2026-11-09