Custom CUDA engine pushes RTX 4090 near Blackwell-level inference results
Distinct-Pie2389 · reddit · 2026-11-20
An RTX 4090 (ADA) user built halocline, a custom CUDA inference engine aiming to deliver near-Blackwell performance on ADA cards. It supports both Dense (27B) and Sparse (QFN) paths, prioritizing quality via lower mean KLD and higher BPW, using Q4KXL with 4-bit rotated KV for the Dense path.
Community benchmarks (airbench.ai):
- QFN: 49/49 in 25 minutes on UD-IQ4XS, ranked #13.
- 27B: 48/49 in 1h29m (only a vision task failed) on UD-Q4KXL, matching NVFP4 quasar 27B results that finish in 4 minutes on an RTX 5090, ranked #65.
Unofficial benchmarks; repo at github.com/T-Crypt/halocline.
Related event: Open-Source CUDA Engine Halocline Boosts RTX 4090 Inference up to 2.6x(2 posts)→
More from Infra
- Microsoft's USB4 RDMA lets mini PCs pool memory to run Qwen3-235B — JFPuget · 2026-11-22
- LTX 2.5 keeps crashing on AMD 9070 XT with driver timeouts while MiniMax H3 runs fine — ZestycloseReading504 · 2026-11-22
- FT: AI server suppliers raise prices up to 45% months after deals are signed — nordicinst · 2026-11-22
- China Installed 59% of New Industrial Robots in 2026 Report, ~354,000 Units — ingliguori · 2026-11-22
- MLX over Vulkan hits 124 tok/s on RTX 5060 Ti, beating llama.cpp CUDA by 25% — SumitGup · 2026-11-22
- Intruder swapped Prime Intellect's hosted DeepSeek for an abliterated copy for 53 hours — mawcopolow · 2026-11-22