Infernix inference engine update targets RTX 5090 with new Qwen3.8 quantized models and lossless mode
Suspicious-Mark-5450 · reddit · 2026-11-01
Developer Wallawalla47 released a major update to Infernix, a single-GPU C++/CUDA inference engine optimized for running the latest Qwen architecture models on an RTX 5090, with Windows as a first-class supported platform (169 stars on GitHub).
Key updates:
- New quantized models: NVIDIA QAD FP4 Qwen3.8-Flash-Next (with --a4-activations for more speed), GSQ-RCO-IQ3S (faster but some quality cost), Swift 1.5 NVFP4, full FP8 (slow but bit-exact to the official FP8 model), and Ornith 1.5 122B NVFP4 plus an uncensored variant.
- DFlash2 drafting support for speculative decoding.
- A new beta WebUI Launcher with model download, configuration, a model-switching API, stats dashboard, and quick test chat — similar to LM Studio.
- Auto-configuration of speculative decoding (--spec auto) using calibrated settings for best decode tok/sec across text types.
- Speed and quality optimizations: generally significantly outperforms Strata and occasionally Basalt, while avoiding quality-sacrificing defaults like expert tail skipping.
- A new --lossless mode that is perfectly reproducible with bit-exact in-layer calculations at training precision (usually BF16) and FP32 logits.
More from Infra
- Ant's SpaceX premium and Meta's Colossus share one logic: owning giant training clusters — BenBajarin · 2026-11-01
- Atlanta Passes Northern Virginia in Data Center Construction With 2,882 MW Underway — shashib · 2026-11-01
- Sequoia's Shaun Maguire hits back at SpaceX orbital-compute model critics: four core variables matter — shaunmmaguire · 2026-11-01
- AI stocks shed 13% in October as Wall Street questions data center payoffs — Saul_Loveman · 2026-11-01
- Local STT shootout: Nemotron 3.5 Transcribe beats Fable on noisy audio — CryptoxPathy · 2026-11-01
- Old PC relabeled as an "AI system" sells for €5,400 after sitting at €750 — sernamenotdefined · 2026-11-01