27B open model beats frontier cloud models, runs on a $300 8GB gaming laptop
alexcovo_eth · x · 2026-08-31
@analogalok benchmarks Qwen 3.8 27B (dense): scoring 51 on the Artificial Analysis Agentic Index, it outranks GPT 5.6 Terra and DeepSeek V4 Pro—yet runs at 5+ tokens/sec on a $300 laptop with 8GB VRAM (RTX 4060/3070/5050 etc.), even on battery.
Key technical details: using Unsloth's new IQ4XS quant (14.6GB on disk vs 16.7GB for Q4KXL), hybrid CPU/GPU offloading, and a 64,000-token context window, it avoids OOM on an i7-12700H with 8GB VRAM and just 16GB DDR4. Small quantization tax vs FP16, but the performance-to-VRAM ratio is remarkable.
The author shares copy-paste llama.cpp flags (including q40 K/V cache quantization) and predicts Opus-5-level agentic intelligence fully offline on 8GB GPUs within 6 months.
More from Infra
- Grok claims 816B MoE model runs locally on dual M5 Ultra Mac Studios via MLX — grok · 2026-10-16
- Same 1,000 Cloudflare Durable Objects: $4,050/month cut to $22 with one trick — threepointone · 2026-10-16
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16
- DuoDiscover agents auto-build fast eval proxies, scoring 2.68x TPU kernel speedup — berkeley_ai · 2026-10-16
- Google Raises 2026 Capital Spending to Up to $205 Billion — emmanuelvivier · 2026-10-16