RTX 5090M runs Gemma 4 31B at up to 4,154 tok/s prompt speed, iNordic shares configs
leonbollerup · reddit · 2026-10-29
Small research lab iNordic (northern Sweden) published an "optimal setup" for running Gemma 4 31B on an RTX 5090M, researched with Claude and their own Sysops AI harness skyclaw, with configs open-sourced.
Key numbers:
- Prompt processing: Fast profile (NVFP4) hits 4,154 tok/s at 8K context, 1,600 tok/s at 130K; the Max-context profile (QAT Q4) sustains 528 tok/s at 231K context.
- Decode: single user with MTP and 7 drafts reaches 118 tok/s in short chats, 110 tok/s after 8K of context, versus 33 tok/s on the abliterated Q4K baseline.
Feedback and improvement suggestions are welcome at github.com/inordic/rtx5090-NInferno-gemma4-31b.
More from Infra
- Baseten makes Qwen3-ASR transcribe hour-long audio in 5 seconds at ~$0.0013/hr — baseten · 2026-10-29
- Google DeepMind engineer shows five CUDA optimizations that speed up 3D Gaussian Splatting by 10.5x — janusch_patas · 2026-10-29
- Snowflake CEO joins the 'calling top' chorus as analysts flag AI's compute leverage risk — SumitGup · 2026-10-29
- Dave White: a US-China AI training slowdown deal is largely enforceable via zk proofs — _Dave__White_ · 2026-10-29
- NVIDIA FastGen ships major update: FLUX.2, LTX-2.5, Cosmos 3 plus four new acceleration methods — ArashVahdat · 2026-10-29
- Decagon details a self-improving autoresearch loop for inference stack optimization — kimberlywtan · 2026-10-29