Qwen3.8-27B EXL3 one-click kit brings quality local LLM to 16-32GB consumer GPUs
udmrzn · x · 2026-09-13
MiaAI Lab released a one-click serving kit for Qwen3.8-27B in turboderp's EXL3 quants, targeting consumer NVIDIA cards with 16-32GB VRAM (RTX 3090, 5060 Ti, 5070 Ti, 5090 and more). The kit auto-picks a quant that fits your VRAM (2.0bpw floor for 16GB), sets up its own Python env, downloads weights, serves an OpenAI-compatible endpoint and opens a chat UI, with identical behavior on Windows and Linux. A physician reports using it locally to build an educational ventilator simulator with an infinite random case generator in hours.
More from Infra
- TSMC reportedly raising CoWoS packaging prices up to 20% next year on AI chip demand — Beth_Kindig · 2026-10-31
- effect-harness on Cloudflare: run agent loops, tools and credentials on separate VMs — samgoodwin89 · 2026-10-31
- Google's Project Suncatcher boots TPU in orbit and beams back first AI response — sundarpichai · 2026-10-31
- Together AI launches Post-Training platform for agentic RL with tool-use environments — togethercompute · 2026-10-31
- General Compute raises $100M for inference capacity as CoreWeave stock hits 12-month low — YvesMulkers · 2026-10-31
- Analyst warns of unstable AI compute feedback loop as Kioxia's NAND price warning triggers flash crash — bookwormengr · 2026-10-31