RustingStrata runs Flint 40B on a 12 GB GPU at 64 tok/s, 33% faster than llama.cpp

Serious_Scarcity7818 · reddit · 2026-11-14

A Reddit user benchmarked RustingStrata against llama.cpp running the same Recursive Flint UD-Q5KXL (27.9 GiB) model on an RTX 3060 12 GB with 30 GB RAM.

Warm decode speed: 64.6 tok/s vs 48.6 tok/s for llama.cpp — about 33% faster; warm 4K prompt prefill of 1.14–1.30k tok/s vs 0.85–1.02k tok/s.
Since the model exceeds VRAM, expert weights must be fetched from system RAM. Two changes drove the gains: adapting expert placement to the reply instead of the prompt's routing pattern, and releasing RAM pages for experts already copied to VRAM while preserving cache for those still pending.
The author ran quality benchmarks and argues RustingStrata loses no quality — the speed comes from model-specific code, while llama.cpp trades speed for generality.
They also suggest Recursive Flint beats Qwen3.6:35-a3b and could serve as a modern replacement.

Full benchmarks in docs/BENCHMARKS.md (RustingStrata 1.2.2, llama.cpp b12191).

Original post →

RustingStrata runs Flint 40B on a 12 GB GPU at 64 tok/s, 33% faster than llama.cpp

More from Infra