Unsloth runs Laya Decision models locally on just 4GB RAM across CPU, Mac and GPU
danielhanchen · x · 2026-09-28
UnslothAI announced that Laya Decision models can now run locally on devices with only 4GB of RAM, working across CPU, Mac, Windows, Linux and GPU setups. Models can be served through a compatible API via Unsloth Desktop. The repo (76.9k stars) is the first desktop app to both run and train local LLMs and diffusion models, supporting GGUF, MLX and many popular model families.
More from Infra
- OriginTrail ships DKG V10.0.19 on mainnet for faster AI agent context graphs — melnykowycz · 2026-09-29
- INT21 claims 20 AI-generated inference engines in 2 weeks, MiMo hits 1,308 tok/s — bingxu_ · 2026-09-29
- Agent-driven synthetic monitoring with Amazon Nova Act replaces brittle UI scripts — AWS ML Blog · 2026-09-28
- NVIDIA demos 2,529 output tokens per second for Qwen 27B at GTC — IanAndrewsDC · 2026-09-28
- Automating Amazon Textract adapter lifecycle management across accounts — AWS ML Blog · 2026-09-28
- Muse is one CPU-heavy implementation, not a proxy for sizing the agentic CPU opportunity — BenBajarin · 2026-09-28