Redditor pleads with FP4 inference engine builders: small dense models at FP4 are cooked
buttplugs4life4me · reddit · 2026-09-20
A Reddit user vented about the flood of daily "fastest inference engine" posts that turn out to be NVFP4/MXFP4-only. The argument: squeezing a bit more speed out of the already-fastest option usually comes with badly degraded outputs and rampant hallucinations. Large models have enough redundancy to survive 4-bit, but small dense models at FP4 are completely killed — the model starts claiming 1+1=3. The author pleads with the community to stop the FP4 arms race.
More from Infra
- Agent Substrate Calculator Estimates Session Resume Speed Under Load — pjausovec · 2026-11-12
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- Potion UI's local login system turns your PC into a multi-user AI server — cocktailpeanut · 2026-11-12
- AWS Finally Built Its Own AWS Emulator — super_linear · 2026-11-12
- Megakernel is multiprogramming for GPUs: 'GPUs are entering the CPU age' — pranjalssh · 2026-11-12