SenseNova-U1.5-8B hits 125 tok/s on M5 Ultra in llama.cpp and spots a $1 receipt error
Binary_orchid · reddit · 2026-10-31
A Reddit user benchmarked SenseNova-U1.5-8B-MoT locally on a Mac Studio M5 Ultra via llama.cpp (support merged Oct 21; understanding-only — image generation needs stable-diffusion.cpp).
Benchmarks (llama-bench, Metal, all layers on GPU)
| Quant | Size | pp512 | tg128 |
|---|---|---|---|
| BF16 | 17.41 GiB | 3398 tok/s | 56 tok/s |
| Q80 | 9.25 GiB | 3582 tok/s | 88 tok/s |
| Q4KM | 5.32 GiB | 3482 tok/s | 125 tok/s |
Vision: given a fake grocery receipt with a $1-inflated subtotal, Q4KM summed the six items to 28.99 vs the printed 29.99 in 16s including load (mostly thinking), and even noticed the 7% tax was computed from the wrong number; Q80 caught it too. It also read menus, a whiteboard, and Chinese/English signs accurately (4-10s each), though those test images were made with the same model.
Gotchas
- mradermacher's quants lack mmproj (no vision); converting your own with converthftogguf.py --mmproj fails with KeyError: 'visionconfig' — fixed by changing line 80 of conversion/sensenovau1.py to visionconfig = self.hparams.get("visionconfig", self.hparams).
- llama-cli is now a TUI; redirecting its output to a file yields nothing. Use llama-mtmd-cli, which prints to stdout.
More from Infra
- Particle-1, a token-efficient model, launches on Cloudflare AI Gateway — garrytan · 2026-10-31
- Alphabet plunges 12.6% on $51.4B capex as AI spending cracks the market — pdamodaran · 2026-10-31
- Strix Halo one-command Docker for vLLM: Qwen3.8-Flash-Next hits 43 tok/s decode — asfbrz96 · 2026-10-31
- AMD reportedly plans another Radeon GPU and GDDR6 price hike in November — chemist_slime · 2026-10-31
- How Different NVIDIA GPUs Change Model Outputs via BF16, FP8, FP4 Precision — Kyrannio · 2026-10-31
- Kubernetes AI Conformance becomes a permanent subproject under SIG Architecture — TerryTangYuan · 2026-10-31