GAFU Uses Generation-Side Representations to Boost Fine-Grained Visual Understanding in Unified Multimodal Models
Tengju Ru · hf · 2026-10-23
GAFU, shared on Hugging Face, leverages native generation-side representations in unified multimodal models to improve fine-grained visual understanding.
- Stage 1 (Compression & Alignment): images pass through a generation encoder to produce G-tokens, compressed into compact tokens via spatial binding (local patch prediction) and relational structure alignment (global image affinity plus multi-scale local region affinity).
- Stage 2 (Task-Conditioned Injection): at inference, projected G-tokens serve as K/V in cross-attention while text embeddings act as Q, enabling task-dependent read-out of fine-grained visual details with residual updates.
This lets unified understanding+generation models retrieve fine-grained visual information on demand, e.g. precisely answering questions about small image details.
More from Multimodal
- Two Midjourney sref codes for grainy pastel fashion illustration style — sergeantsref · 2026-10-23
- Three free ways to try MOSS-Image: API playground, HF Space, ComfyUI — songguoxiansen · 2026-10-23
- Open-source MOSS-Image tops 1M ComfyUI downloads, 5B model rivals closed rivals — songguoxiansen · 2026-10-23
- MiniMax Design MCP goes live: create with Design inside Codex and other agents — Hailuo_AI · 2026-10-23
- Quiver's detailed SVG generation sparks debate: code-drawn images could replace pixel-based image models — stuffyokodraws · 2026-10-23
- RLSplat Cuts Feed-forward 3D Gaussian Splatting Memory 9.6x Linearly — Injae Kim · 2026-10-23