KVARK Flash: a proxy that caps DeepSeek's thinking per request, 30-50% fewer tokens
Ill_Contribution_717 · reddit · 2026-11-19
KVARK Flash 1.0 (Apache 2.0) is a small OpenAI-compatible proxy that sits in front of an unchanged DeepSeek V4.1-Flash and decides per request how long the model may think — no weights touched. A rules-only router picks profiles from tools and text: general (chat/math/science) at effort 0.3; code adds budget forcing, closing thinking past 28,672 tokens; docs stops after three searches with no supporting page (answers NOT IN DOCUMENTS); user-supplied reasoningeffort always wins.
Across 7 public benchmarks (4x H100, SGLang FP8, same machine): GPQA Diamond 91.2→89.6, AIME 2026 95.8→98.3, LiveCodeBench v6 90.7→92.0, MMLU-Pro flat — with 29-49% fewer reasoning tokens and 1.4-1.7x speedups. A mixed 263-task batch: 241→242 correct, wall clock 2,033s→1,270s (1.6x throughput). Results hold on 2x DGX Spark.
Honest caveats: six of seven benchmarks send no tools, so the router mostly lands on general — largely equivalent to a fixed effort=0.3 baseline that was not run. The proxy earns its keep on what benchmarks don't exercise (code cap, docs stop rule); hard competition math can drop 2 points (HMMT −2.4).
More from Infra
- US blocks Chinese firms from renting Nvidia chips abroad while Microsoft nears $500M OpenAI deal for ByteDance — kevinnbass · 2026-11-19
- vikhyatk: we are overinvesting in GPU compilers, underinvesting in CPU compilers — mohbibi_ · 2026-11-19
- Polymarket launches Anthropic IPO perps, ANT trades near $104 ahead of Nasdaq debut — Polymarket · 2026-11-19
- R9700 at 160W for llama.cpp: ~20% pp loss, near-silent fans — LippyBumblebutt · 2026-11-19
- Baseten runs DeepSeek v4.1 Flash at ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-19
- Cloudflare launches Spend Management: auto-pause Workers billing at limits — JeremyCMorgan · 2026-11-19