RustingStrata: pure-Rust GGUF engine runs Inkling 2 Mini at 3.5x llama.cpp write speed on an RTX 3060

Serious_Scarcity7818 · reddit · 2026-11-22

A developer added Thinking Machines' Inkling 2 Mini (26B) support to RustingStrata, a pure-Rust GGUF inference engine, and benchmarked it against llama.cpp on an RTX 3060 12GB.

Supports CUDA, Vulkan and CPU-only with no flags; text-only, greedy only, up to the model's 1M context. Open source on GitHub (RustingStudio/RustingStrata); the author asks for Vulkan numbers from AMD/Intel cards.

Original post →

More from Infra

Infra channel →