Running quantized Inkling on a 5-year-old laptop: pre-AGI perks go off-grid
teortaxesTex · x · 2026-11-21
The poster runs Unsloth's UD-Q8KXL quant of Inkling on a 5-year-old laptop, musing that even old hardware gets pre-AGI perks — with a few TBs of knowledge you could survive off-grid (not that it would help much). He notes the quant is really BF16 dense + NVFP4 experts, so performance on his machine was laughable.
The screenshot shows Inkling-2-Mini reading a DeepSeek 'Code Agent Data Engineer' job posting (Beijing, Model Data Strategy) at 12.4 tok/s and accurately summarizing it: building RL environments from real-world coding/cybersecurity/research scenarios, with Code/Cybersecurity/Science tracks.
More from Infra
- 2M canary tokens expose stolen prompts and compromised AI providers — AccBalanced · 2026-11-21
- Unsloth release runs 26.4B Inkling 2 Mini on 16GB RAM, cuts MCP token use 87% — danielhanchen · 2026-11-21
- Seroter Daily Reading List #914: harnesses are cheap to swap, model behavior isn't — rseroter · 2026-11-21
- NVFP4 MoE experts force CPU fallback in llama.cpp; requantizing boosts Metal speed 5.8x — teortaxesTex · 2026-11-21
- Inkling 2 Mini tops intelligence-per-joule test: 82.3% accuracy at just 1.17 kJ per query — simonguozirui · 2026-11-21
- Compute overbuild skeptic recants: efficiency gains cut AI compute cost ~4x per year — teortaxesTex · 2026-11-21