Open-source recipe: abliterated DeepSeek-V4.1-Flash at ~200 tok/s on 4x RTX PRO 6000
burny_tech · x · 2026-10-26
Jina AI founder Han Xiao open-sourced a deploy recipe for the abliterated DeepSeek-V4.1-Flash-1017 on GCP: 4x NVIDIA RTX PRO 6000, FP8 weights (native FP8 E4M3 dense with UE8M0 block scales), LiL vLLM + DSpark, hitting 166–211 tok/s single-stream and 722 tok/s at 8 streams, with 1M context, vision, tools, and spot-rescue. The repo ships full GCP scripts, configs, and patches. He notes nothing has been hillclimbed yet.
More from Infra
- Micron opens second advanced packaging plant in Houli, a concrete HBM step — tengyanAI · 2026-10-26
- What do you do with the spare 8 GPUs in a 64-card NVL72 setup? Keep them hot as spares — _xjdr · 2026-10-26
- Jalapeño chip is just a systolic array with HBM: why custom inference silicon is easier than you think — ai · 2026-10-26
- Torch.compile Flag Cut 16 Regions to 2, Delivering 4.53x Speedup on B200 — cfregly · 2026-10-26
- Dev gets Qwen3.8-Flash-Next W4A16 tuned and running on a 128GB RTX Spark N1X, open-sources the setup — mycall · 2026-10-26
- Jassy: $350M Annapurna deal is Amazon's best acquisition; chip business at $25B run rate — SumitGup · 2026-10-26