Custom CUDA engine pushes RTX 4090 near Blackwell-level inference results

Distinct-Pie2389 · reddit · 2026-11-20

An RTX 4090 (ADA) user built halocline, a custom CUDA inference engine aiming to deliver near-Blackwell performance on ADA cards. It supports both Dense (27B) and Sparse (QFN) paths, prioritizing quality via lower mean KLD and higher BPW, using Q4KXL with 4-bit rotated KV for the Dense path.

Community benchmarks (airbench.ai):

Unofficial benchmarks; repo at github.com/T-Crypt/halocline.

Related event: Open-Source CUDA Engine Halocline Boosts RTX 4090 Inference up to 2.6x(2 posts)→

Original post →

More from Infra

Infra channel →