WeeLLM: Run FLUX.1-dev on 4GB VRAM Without Quantization
AlarmingPhrase8174 · reddit · 2026-08-16
The author released WeeLLM, a layer-streaming inference engine that runs large image models on low-VRAM devices without quantization. By streaming transformer layers one by one from disk, it runs the 12B FLUX.1-dev on an RTX 3050 (4GB) with just 1.51GB peak VRAM. It supports 24 models but requires NVMe SSD speeds and currently only supports text-to-image.
More from coding & agent
- Self Labs launches Agent Vault to let AI agents log in and pay with credit cards — HeyAmit_ · 2026-10-16
- ERP•AI launches Proto: business software built for agents, 'Headless SaaS' — HeyAmit_ · 2026-10-16
- Coding agents keep opening 3 stacked PRs per chat, and merging them is a nightmare — holdenmatt · 2026-10-16
- Google-led paper uses sequential auctions to organize decentralized LLM agent workflows — _onionesque · 2026-10-16
- Since OpenAI's math release, GitHub contributors keep driving down the integer-multiplication kappa — xennygrimmato_ · 2026-10-16
- Pattern MCP makes coding agents check existing components before writing new code — DonR954 · 2026-10-16