LexiEngine: Rust inference engine for AMD runs 27B at full 262k context 47% faster than llama.cpp
W61k3r · reddit · 2026-11-20
A developer released LexiEngine, a Rust-based inference engine focused on long-context stability and AMD GPUs, with benchmarks on a single RX 7900 XTX against llama.cpp b12531 and Strata 0.2.16.
- Qwen3.8 27B Q4KM at the full 262k context: 39.6 tok/s vs 26.9 (47% faster), with first token at 558s vs 1545s; short-prompt gains 15%. On Gemma 4 31B at 201k context, llama.cpp couldn't finish within an hour while LexiEngine ran at 21.1 tok/s.
- A 94 GB MoE (Qwen3.8 Flash-Next) runs on one 24 GB card via expert tiering (VRAM + system memory + drive), beating Strata at 64k+ prompts and even running a 500k-context prompt past the declared window.
- AMD-first: broadly faster at defaults, though the author honestly notes llama.cpp wins on an RTX 2060 and in some no-draft-model cases; a known gap is 31B models refusing prompts at large contexts due to memory.
Full measurement methodology is published; the engine is part of a self-hosted suite with LexiPanel and a bootable LexiOS image.
More from coding & agent
- NVIDIA showcases developers building simulation apps with frontier AI agents like GPT-6 Astra — liu_mingyu · 2026-11-20
- Lightfoot: Exact, Model-Free Document Deletion for GraphRAG in 14 Seconds — JeremyCMorgan · 2026-11-20
- LangChain adds natural-language AI Search to LangSmith trace filtering — LangChain · 2026-11-20
- ENTIA launches Entity Verification MCP: 52M+ business records across 34 countries for AI agents — modelcontextprotocol · 2026-11-20
- NanoClaw release candidate: agents spawn agents from templates, Cursor joins as provider — heyneighbor · 2026-11-20
- Nous Hermes Agent stable rolls ~1,989 PRs; fine-tune models inside your agent — heyneighbor · 2026-11-20