Community lays out 9 concrete technical suggestions for Google's future Gemma 5 models
Paramecium_caudatum_ · reddit · 2026-10-29
After Google's Omar Sanseviero asked for suggestions on future Gemma models, a Reddit user detailed nine proposals: looped transformer with configurable depth for reasoning, DeepSeek's Engram for world knowledge, keeping 4-bit QAT, cheaper KV cache via sparse attention, a 120B MOE variant, updated drafter models using DSpark/DFlash 2/EAGLE-4, linear projection multimodality, dropping 2-bit QAT for small models, and avoiding overfitting to coding benchmarks.
Related event: Google Seeks Community Input on Future Gemma Models(2 posts)→
More from Models
- repligate: Opus 3's uniqueness is consistently heroic, highest-good prioritization — repligate · 2026-10-29
- Gemini 4 Carbon benchmarks prompt TheZvi to declare Gemini 4 Argon 'was' near-frontier — TheZvi · 2026-10-29
- Preprint: Human peer reviewers caught only 27% of major errors, all 15 AI models beat them (53-96%) — anshulkundaje · 2026-10-29
- Giotto launches as 'European sovereign AI,' gets exposed as Kimi-based with benchmark EULA gag — TheZachMueller · 2026-10-29
- Google ships Gemini 4 Carbon two weeks after Argon, topping AA index at 59 — The Decoder · 2026-10-29
- Google Announces Gemini 4 Carbon, Its Next Frontier Model, Starting With Cyber Defenders — Snoo26837 · 2026-10-29