DeepSeek V4.1 Flash EXL3 2.81bpw quant lands with 57tps decode, 60-90% speedups
HankYeomans · x · 2026-11-02
A community EXL3 quantization of DeepSeek V4.1 Flash at 2.81bpw is now available, quantized with exllama3's self-corrected experimental branch.
- Slightly smaller than MiaAIlab's EXL3 EAQ v1 2.9bpw build.
- With custom TurboEXL3 fused kernels for streaming, it hits 57tps decode and 1400tps prefill — roughly 60% and 90% faster than the author's previous version; further gains expected once released in mlx-serve.
- Weights are on Hugging Face with a chat template transcribed for jinja.cpp, supporting thinkingmode (chat/thinking) and reasoningeffort (low/high/max or 1-100) parameters.
More from Infra
- AI data centers destroy 250,000 trees, undercutting 'paperless' eco promises — im_mansigupta · 2026-11-02
- "Someone must solve the cost of serving coworking agents" — hyhieu226 · 2026-11-02
- SmolChat v17: On-Device GGUF LLMs on Android Now Ships an OpenAI-Compatible Server — shubham0204_dev · 2026-11-02
- 8 Coding Agents × 250K Context Each, Concurrently, on Two RTX 3090s (~600+ tok/s) — o2beast · 2026-11-02
- Liquid AI's LFM3 Omni 2B Runs Fully In-Browser via WebGPU with Sub-0.3s Voice Latency — rm-rf-rm · 2026-11-02
- Google Runs Gemma on TPU in Orbit, First Step Toward Sun-Powered Space Data Centers — shashib · 2026-11-02