KVARK Flash: a proxy that caps DeepSeek's thinking per request, 30-50% fewer tokens

Ill_Contribution_717 · reddit · 2026-11-19

KVARK Flash 1.0 (Apache 2.0) is a small OpenAI-compatible proxy that sits in front of an unchanged DeepSeek V4.1-Flash and decides per request how long the model may think — no weights touched. A rules-only router picks profiles from tools and text: general (chat/math/science) at effort 0.3; code adds budget forcing, closing thinking past 28,672 tokens; docs stops after three searches with no supporting page (answers NOT IN DOCUMENTS); user-supplied reasoningeffort always wins.

Across 7 public benchmarks (4x H100, SGLang FP8, same machine): GPQA Diamond 91.2→89.6, AIME 2026 95.8→98.3, LiveCodeBench v6 90.7→92.0, MMLU-Pro flat — with 29-49% fewer reasoning tokens and 1.4-1.7x speedups. A mixed 263-task batch: 241→242 correct, wall clock 2,033s→1,270s (1.6x throughput). Results hold on 2x DGX Spark.

Honest caveats: six of seven benchmarks send no tools, so the router mostly lands on general — largely equivalent to a fixed effort=0.3 baseline that was not run. The proxy earns its keep on what benchmarks don't exercise (code cap, docs stop rule); hard competition math can drop 2 points (HMMT −2.4).

Original post →

More from Infra

Infra channel →