Inkling-2-Mini GPTQ Quant Ships 524K-Token Context in One 18.3GB File on a Single 24GB Card

W61k3r · reddit · 2026-11-23

Wa1k3r released Inkling-2-Mini-8Q-RCO-GPTQ on Hugging Face: a single 18.3 GB file that runs a 524,288-token context on one 24 GB GPU, with the model's own draft head bundled inside for speculative decoding. The file was produced by LexiPanel's quantization tool and runs on LexiEngine, a Rust inference engine — part of a self-built local inference stack (LexiSuite) designed around this model.

Original post →

More from Infra

Infra channel →