QLoRA author: DeepSeek's open inference stack rivals OpenAI's and beats Anthropic's
DavidBennett__ · x · 2026-10-21
- Tim Dettmers, the QLoRA author, says DeepSeek's newly open-sourced inference system is on par with OpenAI's internal stack and better than what Anthropic runs at their user scale.
- It ships under Apache 2.0 and runs on both NVIDIA Blackwell and Huawei Ascend, avoiding lock-in to a single chip vendor.
- Production stacks of this caliber previously lived inside a handful of labs — the invisible source of their API speed and fat margins. Now any startup, university, or cloud can download, read, and run it.
- The takeaway: open weights are about more than chatbots; DeepSeek is giving away knowledge only a few labs had, changing who can build an AI company at all.
More from Infra
- Uncensored DeepSeek V4.1 Flash GGUFs drop refusal rate from 91.4% to 0% with minimal MMLU loss — kalyan_kpl · 2026-10-21
- Specter speeds single-GPU MoE inference with quantized multi-step speculative offloading — Wenfeng Wang · 2026-10-21
- SpaceX reportedly targeting 13 GWh of compute by EOY 2027; Grok bots for businesses in weeks — elonmusk · 2026-10-21
- Meta engineer confirms Google TPUs now run AI inference in Meta production — dotey · 2026-10-21
- OmniGraph Open-Sourced: Git-Style Branches and Merges for AI Agent Memory — techNmak · 2026-10-21
- Running Qwen3.8 for agentic coding on 2x5090: vLLM is the only stack that doesn't trash the cache — TooManyPascals · 2026-10-21