Dev reimplements DeepSeek v4.1 Flash, runs 1M context at 1-10 tok/s on a single RTX 4090
_xjdr · x · 2026-09-18
Developer xjdr implemented DeepSeek v4.1 Flash from scratch to understand the architecture, getting it to run at 1-10 tok/s (bs=1, ctx=1M+) at full precision (bf16/fp8/mxfp4 mix) on a single RTX 4090 with 64GB RAM and 1TB NVMe, depending on expert cache hits. His verdict: the architecture 'slaps' but is optimal for Grace Blackwell-class deployments, and he's impressed by DeepSeek's engineering under hardware constraints.
More from Infra
- Fire at Taiwan Panasonic PCB materials plant may tighten CCL supply further — zephyr_z9 · 2026-11-04
- Same Claude model served by Bedrock one day and Anthropic the next, dev finds — georgos_pappas · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Gigabyte's AI TOP motherboard packs 2TB RAM and 9 GPUs for home AI labs — vincent_koc · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04
- imec neural probe integrates photonics directly into 130nm CMOS backend — jwt0625 · 2026-11-04