capy-flash v0.16 runs Inkling 2 Mini on two 16GB GPUs at 149 tok/s with 254k context in NVFP4

FeydRowan · reddit · 2026-11-22

Developer Valerio Dolci released capy-flash v0.16, adding Inkling 2 Mini support to his inference engine (alongside Qwen3.8-27B and Qwen3.8-Flash-Next), optimized for tensor parallelism on two 16GB Blackwell cards on Linux; single 5090 and Windows via WSL2/Docker should work but are untested.

Benchmarks (2× RTX 5070 Ti 16GB, from the infographic):

The author plans targeted quantization for speed after measuring current performance. Code and weights are open on GitHub and Hugging Face.

Original post →

More from Infra

Infra channel →