9B model hits 59 tok/s on a 16GB GPU, but tool calls lag at 58% BFCL
AIFlow_ML · x · 2026-11-22
JJ Eaton ran SenseNova-Loop-9B on TensorFold: 59.1 tok/s on an RTX 5070 Ti and 59.5 tok/s on an M5 Ultra, on a 16GB GPU.
- The weak spot he flagged himself: tool calls, 58% on BFCL
- Verdict: fine for chat on a cheap card, not yet your agent brain
More from Infra
- SK Hynix reportedly tops 70% yield on HBM4, starting mass production in Q3 — Beth_Kindig · 2026-11-22
- Mflux Runs MOSS Image Gen on 8GB Macs: 82s for 768x768 With 4-Step Distill LoRA — bio_risk · 2026-11-22
- After a Possible Riemann Proof, a Call to Reinvent AI Memory and Inference Cost — Kitchen_Mirror_7247 · 2026-11-22
- Intel D270 AI card: 160GB LPDDR5X, 350W, ~$10K, and why call it a GPU? — Terminator857 · 2026-11-22
- Speed-to-power dominates: on-site solar-plus-storage cheapest for AI data centers in 85% of US locations — mattbeane · 2026-11-22
- LexiEngine, an AMD-focused Rust inference engine, adds Inkling support — W61k3r · 2026-11-22