2-bit DeepSeek V4.1 Flash runs a coding-agent seat on two 2016 V100s: a 12-hour real-work trial

a4mula · reddit · 2026-11-13

A Reddit user let Claude Sonnet 5.5 mentor a locally-hosted DeepSeek V4.1 Flash (1017, MIT license) as a candidate replacement in the Systems Engineer seat of their single-workstation AI lab, running a 12-hour unattended evaluation on real project work instead of benchmarks.

Setup highlights

Quantization: ISTA-DASLab GSQ-RCO IQ2XXS at 2.00 bits/weight, 6-shard GGUF, 131k context.
Engine: MIT-licensed Strata, built for experimental Volta support with zero source patches; the 132 GB of experts stream from system RAM against only 64 GB of VRAM, with a hot-expert pre-seed profile.
Hardware: 2x Tesla V100 32GB (no BF16, no FA2, no NVLink), dual Xeon, 125 GB RAM.
Measured: 24 tok/s decode, 127 tok/s prefill, 9s to first token; the speculative drafter actually hurt (17.3 vs 23.8 tok/s) since experts don't fit in VRAM.

Verdict: DS worked like a competent junior — clean commits, sandbox proof-checkers, DB migrations, disciplined evidence logging. But latency is brutal (17s per message, 5 min per web fetch, 7 min per compaction). The biggest insight: observing where DeepSeek stumbled exposed how many small issues Sonnet quietly brute-forces past in their own system.

Original post →

2-bit DeepSeek V4.1 Flash runs a coding-agent seat on two 2016 V100s: a 12-hour real-work trial

More from coding & agent