Gemma 4 E2B traced to bidirectional Gemma 4 continuation, not T5Gemma-2
tomaarsen · x · 2026-10-29
Responding to questions about the google/gemma-4-E2B-it architecture, tomaarsen explains that it is not based on the T5Gemma-2 encoder at all — it's more a continuation of the Gemma 4 family turned bidirectional.
Inspecting the transformers architecture files, the "modular" file (used to generate the runnable "modeling" file) shows heavy code reuse from Gemma 4 and Gemma 3, suggesting Google may have done some layer trimming since existing T5Gemma checkpoints are too large to continue from.
Related event: EmbeddingGemma2 Revealed as Bidirectional Gemma 4 Successor(3 posts)→
More from Models
- Zvi calls out Google's RSI contradiction: execs tout it while the safety card denies doing it — TheZvi · 2026-10-29
- Reported Gemini 4 Carbon launch tops AQI at 59; Polymarket gives Google 97.5% odds for best October model — vedantmisra · 2026-10-29
- repligate: Opus 3 has done the most good of any model, with Opus 4.5 second — repligate · 2026-10-29
- Gemini 4 Carbon benchmarks prompt TheZvi to declare Gemini 4 Argon 'was' near-frontier — TheZvi · 2026-10-29
- Preprint: Human peer reviewers caught only 27% of major errors, all 15 AI models beat them (53-96%) — anshulkundaje · 2026-10-29
- Giotto launches as 'European sovereign AI,' gets exposed as Kimi-based with benchmark EULA gag — TheZachMueller · 2026-10-29