Hermes adds a local backend with Unsloth UD-Q4 quants for DeepSeek-V4-Flash and Qwen models
maximelabonne · x · 2026-09-05
Hermes now has a local backend supporting Unsloth UD-Q4KXL and UD-Q4KM quants for DeepSeek-V4-Flash, Qwen3.8-27B, Qwen3.6-35B-A3B, and Qwen3.8-Flash-Next. Maxime Labonne responded suggesting going even smaller, hinting more compact local options may follow.
Related event: Hermes adds local backend for one-click Unsloth GGUF models(2 posts)→
More from Infra
- Railway pauses all new builds amid major outage, users ask if vibe coding is to blame — AaronBergman18 · 2026-10-21
- a16z: Superhuman AI hardware design is the easy part — manufacturing is the hard one — a16z · 2026-10-21
- Sizing a 1 GW datacenter: ~522,000 Blackwell GPUs fit inside — bookwormengr · 2026-10-21
- DeepSeek's production density reportedly hits ~$38 per GPU-hour at saturation — bookwormengr · 2026-10-21
- PyTorch details KTIR and Dataflow Scheduler behind IBM's Spyre accelerator backend — PyTorch · 2026-10-21
- Indie dev replaces $1k OpenAI bill with a YAML pipeline on spot GPUs and vLLM — cmm324 · 2026-10-21