Replacing 32B with 4B: Extreme Compression for MiniMax H3 Video Generation
Fit_Ad7343 · reddit · 2026-08-09
The author details a method to replace the native Qwen3-VL-32B text encoder in MiniMax H3 with a Qwen3-VL-4B model. By learning a linear projection matrix using ridge regression, the hidden states of the 4B model are mapped into the 32B's conditioning space, slashing VRAM usage from 15.7GB to 4.5GB.
Technical Details & Results:
- Since both models share the same tokenizer, positional alignment is achieved via simple ridge regression without complex gradient training.
- Despite a low test cosine similarity of 0.71, the DiT model tolerates the error well, and the generated video quality remains largely unaffected.
- Includes rigorous control group tests (zero and identity matrices) proving the learned matrix provides crucial structural information.
Limitations:
- World knowledge from the 32B model is lost; certain real people, landmarks, or brands may not render correctly.
- The linear projection has hit its performance ceiling, requiring an MLP for further gains.
More from Multimodal
- ComfyUI Extension Update v2.4 Brings Full Support for Minimax H3 — Obvious_Set5239 · 2026-08-09
- Alleged New GPT Image Model 'mona-lisa-1' Surfaces on Chatbot Arena — mark_k · 2026-08-09
- Workflow Sharing: Building a Consistent AI Character with Multimodal Tools — Gocciole · 2026-08-09
- MiniMax H3 Test: Seamless Video-to-Video Character Replacement — beatlepol · 2026-08-09
- I made a Seinfeld-style short with Minimax H3, locally generated — Altruistic_Dealer_59 · 2026-08-09
- Developer Tests Voice-to-Game Concept Art Workflow, Paises Imagen's Stunning Output — Dimillian · 2026-08-09