Dev wrote Marlin-style FP4/FP8 kernels for RTX 3090 — NVIDIA declined to upstream them
QuixiAI · x · 2026-09-07
Developer buythedip revealed they wrote Ampere Marlin-style FP4/FP8-to-FP16 dequant/GEMM kernels for RTX 3090s, based on QuixiAI's work — and NVIDIA wouldn't upstream the contribution.
Notable for anyone running low-precision quantized inference on older consumer GPUs: community-written kernels fill the gap on Ampere, but the lack of official maintenance means users must maintain their own forks.
More from Infra
- Anthropic locked into up to $120.5B SpaceX compute deal — $3.1B per month from Dec 2026 — ns123abc · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Running a local 9B as a coding agent: brief the big model to plan, let the small model execute — Electrical_Ninja3805 · 2026-10-27