Hermes adds local backend: run Unsloth UD-Q4 quants of DeepSeek-V4-Flash and Qwen3.8 one-click
danielhanchen · x · 2026-09-04
Unsloth GGUFs can now be run locally in one click via Hermes. Daniel Hanchen confirms Hermes has a new local backend with support for Unsloth UD-Q4KXL and UD-Q4KM quants of DeepSeek-V4-Flash, Qwen3.8-27B, 3.6-35B-A3B, and Qwen3.8-Flash-Next.
Related event: Hermes adds local backend for one-click Unsloth GGUF models(2 posts)→
More from Infra
- SGLang merges megakernel low-latency mode, hitting ~1000 tok/s/user on GB300 NVL72 — ying11231 · 2026-10-21
- The topology of an LLM agent: where the model, loop, and tools run shapes cost and security — davidcrawshaw · 2026-10-21
- Chris Lattner shows how MAX and Mojo extend PyTorch across vendor hardware at PyTorchCon 2026 — PyTorch · 2026-10-21
- General Compute partners with Positron AI for cluster-scale deployment, live Q1 2027 — eptwts · 2026-10-21
- MemTensor's MemT-1.0 makes persistent memory part of model compute: 1.86x faster inference, 66x less storage — _akhaliq · 2026-10-21
- LithosAI sweeps 12 #1 spots on Artificial Analysis, GLM 5.3 Flash at 1,543 tok/s — JiaZhihao · 2026-10-21