ComfyUI node expands prompts with a local LLM that offloads VRAM after each generation
ProcedureFit789 · reddit · 2026-11-03
A Reddit user open-sourced a custom ComfyUI node that expands short prompts into detailed text-to-image prompts using a small local LLM — without permanently hogging the VRAM your diffusion models need.
Key design
- Custom inference engine in PyTorch + Triton with INT8 quantization and custom matmul kernels
- Supports Qwen3.5 checkpoints in W8A16 (0.8B) and W4A16 (2B)
- Two memory modes: offload to pinned system RAM after generation (much faster reloads than disk) or keep resident on GPU for reuse
Tested on a laptop with RTX 2050 (4GB VRAM) + 16GB RAM. Caveats: offloading doesn't remove the system RAM requirement (diffusion model, text encoder, VAE plus the LLM's pinned copy still need headroom); on 4GB GPUs use offload mode. Small models may need prompt regeneration or tweaks. Install details in the GitHub README.
More from coding & agent
- Two gaps every AI agent team must understand: requirement gap and model gap — bigdata · 2026-11-03
- Hermes Agent + MeshyAI turns conversational prompts into 3D game characters — egeberkina · 2026-11-03
- First hands-on: Meta's Muse Wonder 1.0 vs DeepSeek V4.1 Flash in Codex CLI — Richidm · 2026-11-03
- "Claude Code has been terrible today" — developer smells a new Opus launch coming — hermanschutte · 2026-11-03
- One command audits every skill loaded by Claude Code, Codex, Cursor and more — 236 found — UnicornAdoption · 2026-11-03
- When Agent Swarms Rewrite Your Backend in a Language You Don't Know — smlpth · 2026-11-03