llama.cpp adds TENSOR_READ_LAZY, MoE expert tensors no longer need to sit in VRAM
jacek2023 · reddit · 2026-08-27
A new llama.cpp PR #27794 by ngxson introduces a TENSORREADLAZY option in the model loader. The key benefit: for MoE models like Qwen3 Next, expert tensors (engrams) that are only needed when activated no longer have to stay resident in VRAM/RAM — they're loaded lazily on demand, substantially lowering the memory bar for running large sparse models locally.
More from coding & agent
- Beyond the model: what's in your agent setup? — External-Wind-5273 · 2026-08-27
- Codex users would trade 5% quota to let running tasks finish — vinmi · 2026-08-27
- qmax-mcp: local QA MCP server that scans URLs and runs Playwright repros for coding agents — bestofdesp · 2026-08-27
- MCP client SDK for Apple's on-device model dynamically builds Tools from JSON Schema — AdventurousKeys · 2026-08-27
- Remotion Studio implements WebMCP for contextual agent actions — Vjeux · 2026-08-27
- What Do Agents Do When They Hit a Raw Table Nobody Has Mapped? No Shared Keys — No-Plant-5234 · 2026-08-27