Tensorfold hits 40-60 tok/s on Qwen3.8-27B with Mac mini M5 Pro, beating MLX
AdRepulsive7837 · reddit · 2026-09-30
A user reports the open-source inference engine Tensorfold reaches 40-60 tok/s running Qwen3.8-27B (4-bit, with a DFlash2 drafting model) on a Mac mini M5 Pro 64GB — the first Mac engine to beat MLX in their tests of omlx, dflash2, mlx, llama-cpp, lm-studio and unsloth. Their RTX-3090ti manages 50-70 tok/s with 4x faster prefill, and they now consider replacing the GPU entirely.
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16