Custom engine runs Inkling 2 Mini at 129.5 tok/s with 344K context on a single RTX 3090, 2.3x faster than llama.cpp

teortaxesTex · x · 2026-11-21

Developer superalesha open-sourced maya-ik2, a Project Maya-based inference engine that pushes Inkling 2 Mini to 129.5 tok/s decode with 344K context on a single RTX 3090 — 2.3x faster decode and 3.7x faster prefill than llama.cpp with zero quality loss.

Key techniques:

The repo is MIT-licensed and supports images and tools.

Original post →

More from Infra

Infra channel →