Gemma 4 Tech Report: Open-Source Native Multimodal Models
burkov · x · 2026-07-08
Google released the Gemma 4 technical report, introducing a new generation of open-source, natively multimodal language models. The Gemma 4 series features both dense and Mixture-of-Experts (MoE) architectures, with parameter sizes ranging from 2.3B to 31B, aiming to boost computational efficiency and reasoning capabilities.
All model sizes feature improved vision and audio encoders. Notably, the 12B model adopts a unified encoder-free architecture, allowing it to process raw audio and image patches directly. Additionally, it integrates a thinking mode, enabling the model to generate a chain of thought before answering.
More from Models
- ICML 2026 position paper: LLM 'thinking traces' are a harmful anthropomorphism — rao2z · 2026-10-27
- One stray 'u' and Claude starts spelling like a Brit — cantrell · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- ARC Prize redesigns leaderboard: highest reasoning effort by default, CSV export — mhmazur · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27
- Model refuses wrong psychiatric advice in deliberately misleading test — FellMentKE · 2026-10-27