Hacked DFlash drafter runs on stock llama.cpp, +61% code speedup for Ling-3.0-tiny
Barni275 · reddit · 2026-10-23
When the newly released UBTree DFlash drafter for Ling-3.0-tiny had no inference engine support, a local-LLM user extracted the pure DFlash part (dropping 7 ubtreeselector tensors) and got it running on stock mainline llama.cpp.
- Measured gains: prose 78.8→84.8 tok/s (+8%, acceptance 0.28); Python codegen 79.0→127.2 tok/s (+61%, acceptance 0.53)
- Method: only tiny patches to the llama.cpp GGUF converter, everything else stock; converted model and procedure published on Hugging Face (yury-kapoyko/Ling-3.0-tiny-UBTree-dflash-GGUF)
- Outlook: native UBTree support in llama.cpp should yield further speedup
The codegen win is far larger because draft acceptance doubles (0.53 vs 0.28).
More from Infra
- x87 floating point in 1995-2005 PC games murders ARM emulators — FEX's new stack optimization pass — lauriewired · 2026-10-23
- Video Model API Markups Exposed: Runway Charges 414%, New Sume API Launches With 0% Markup — Scobleizer · 2026-10-23
- Google Cloud launches Agent Compute Engine, a new VM type built for AI agents — testingcatalog · 2026-10-23
- LM Studio ships Read Aloud, powered by tiny local TTS model Phonon from Gradium — mattturck · 2026-10-23
- Lam Research system revenue up 35% QoQ as AI pushes 2027 WFE past $200B — BenBajarin · 2026-10-23
- Hugging Face Hub launches OpenUSD support with 2,500+ NVIDIA SimReady assets — mishig25 · 2026-10-23