New ComfyUI Node Cuts Flux2 Klein's Text Encoder VRAM From 5.5 to 3.2 GB, Bit-Identical Output
Occsan · reddit · 2026-10-28
Flux2 Klein only reads three hidden layers (9, 18, 27) from its Qwen3 text encoder, but standard loaders build the full 36-layer model plus final norm and LM head—and ComfyUI-GGUF dequantizes the whole token embedding table to fp16. The author's new 'Klein CLIP Loader (Pruned)' node:
- Builds only 27 layers, no final norm or LM head
- Keeps GGUF token embeddings quantized, dequantizing only your prompt's rows
- Produces bit-identical conditioning (max abs diff 0.0 vs CLIPLoader GGUF)
Benchmarks (Qwen3-8B Q4KM, Klein 9B, RTX 4090, 512-token prompt): TE weights on GPU drop 5.51→3.20 GB, encode peak 5.97→3.62 GB—freeing 2.3 GB for the DiT, LoRAs, or a co-running LLM. On 8 GB cards it can decide whether offloading is needed.
Install: clone the repo into ComfyUI/customnodes and swap the loader (requires ComfyUI-GGUF). Side finding: ComfyUI picks models to unload by reference count and size rather than measured reload time; a PR ranking by reload time cut a Klein + local LLM workflow from 13.6s to 6.7s per run (not yet merged).
More from Infra
- preowned turns macOS built-in Apple models into an OpenAI-compatible local API — pcuenq · 2026-10-28
- 2,100+ tok/s/user for DeepSeek V4.1 Flash on 8× AMD MI355X, 2.8x faster than open-source — JiaZhihao · 2026-10-28
- Half of Agent Infra users are no longer human: Cloudflare, Vercel and Supabase numbers — FinanceYF5 · 2026-10-28
- Phonon on-device TTS hits 80K downloads with PyTorch-free Rust runtime — mattturck · 2026-10-28
- Agent infra as the sure bet: Manus runs 1.5M databases on TiDB, investors say — FinanceYF5 · 2026-10-28
- Octen-WebGR Indexes 201M Web Pages in a 4B Model's Parameters, Beats Dense Retrieval and Frontier LLMs at Navigational Search — _reachsumit · 2026-10-28