LexiSuite: a Rust-built inference engine beats llama.cpp 1.0.0 at 262k context
W61k3r · reddit · 2026-11-14
A developer released LexiSuite — a 1.93M-line Rust codebase containing an inference engine, control panel, appliance OS and kernel, not a llama.cpp wrapper. Benchmarked the same day llama.cpp 1.0.0 shipped, running Qwen 27B (Q4KM) at 262k context on a single 24GB RX 7900 XTX:
- Generation 17-25% faster from short prompts up to 33k tokens; prefill 26-37% faster
- First token latency 0.19s vs 0.34s on short prompts
- At 133k and 262k token contexts llama.cpp crashes or returns nothing, while LexiEngine still generates at 50.4/39.9 tok/s
The author uses it daily as a personal AI box and publishes losing cases alongside wins.
More from Infra
- Running DeepSeek 4.1 Flash on a 32GB Mac via SSD streaming with open-source Local Code — alexcovo_eth · 2026-11-14
- SGLang Summit talk details battle-tested NVFP4 4-bit RL across thousands of GPUs — ericzelikman · 2026-11-14
- Will local models keep up with frontier? Users debate a hardware-bound gap — AshAtlasDev · 2026-11-14
- Anthropic IPO roadshow slide leak sparks Blackwell training speculation — and a block — teortaxesTex · 2026-11-14
- New Yorker interview: AI is a brute-force problem that will 'cook the planet' — kevinnbass · 2026-11-14
- Lenore unveils hybrid Token factory in Zhangjiakou: 15,000P cluster, 2 quadrillion tokens a year — 量子位 · 2026-11-14