Replacing 32B with 4B: Extreme Compression for MiniMax H3 Video Generation
Fit_Ad7343 · reddit · 2026-08-09
The author details a method to replace the native Qwen3-VL-32B text encoder in MiniMax H3 with a Qwen3-VL-4B model. By learning a linear projection matrix using ridge regression, the hidden states of the 4B model are mapped into the 32B's conditioning space, slashing VRAM usage from 15.7GB to 4.5GB.
Technical Details & Results:
- Since both models share the same tokenizer, positional alignment is achieved via simple ridge regression without complex gradient training.
- Despite a low test cosine similarity of 0.71, the DiT model tolerates the error well, and the generated video quality remains largely unaffected.
- Includes rigorous control group tests (zero and identity matrices) proving the learned matrix provides crucial structural information.
Limitations:
- World knowledge from the 32B model is lost; certain real people, landmarks, or brands may not render correctly.
- The linear projection has hit its performance ceiling, requiring an MLP for further gains.
Related event: Extreme Compression: Swapping 32B Encoder for 4B in MiniMax H3(3 posts)→
More from Multimodal
- Kling 4.0 fully available, shows markedly better motion coherence in fast-action scenes — alifcoder · 2026-10-16
- MJev-Spatial: Single-Pass Classification Beats Autoregressive VLMs on Spatial Reasoning — mindlab-research · 2026-10-16
- H-DDCM: Hadamard Codebooks Make Diffusion-Based Image and Video Compression Scalable — Technion · 2026-10-16
- ComfyUI v0.41.0 ships with Flux3 image model and MiniMax-H3 LightVAE support — Fcking_Chuck · 2026-10-16
- AI-generated short film 'The Dragon Rider' glides cinematically above the clouds — jijo6686 · 2026-10-16
- Realistic candid Midjourney portrait prompt: natural skin, lived-in bedroom light — tisch_eins · 2026-10-16