Monadize fine-tune fixes Gemma's 'Elara problem': 60 unique names in 30 generations vs heavy repetition on base model
zoraruha · reddit · 2026-10-25
Developer zoraruha released Monadize, a fine-tune of Gemma 4 31B targeting repetitive name generation (the notorious 'Elara/Elias' problem).
- Approach: instead of prompting out common names or banning tokens, it shifts the next-token probability distribution directly, using single-character tokenization when introducing new names; distribution plots show a much wider spread of starting letters.
- Results: at temperature 1, 30 character names + 30 city names: the base model repeated heavily (Elias Thorne ×10, Aethelgard ×15), while Monadize produced 60 unique names.
- Side effect: storytelling style also diverged noticeably, with same-seed short story comparisons included.
Open-sourced at huggingface.co/zoracapri/gemma-4-31B-Monadize-v0.1.
More from Models
- Drex 2.0 goes fully local on Apple Silicon: open weights, 113ms median latency — flngr · 2026-10-25
- True-or-false quiz: 5 common misconceptions about LLM tokens and context limits — ClickOk5811 · 2026-10-25
- Leak: OpenAI quietly restarted frontier training Oct 1 with offline web; agents found Brotli RCE — mark_k · 2026-10-25
- Claude weekly usage limit hit 96% as power user begs Anthropic for relief — adonis_singh · 2026-10-25
- ChatGPT voice mode spotted using open-source Silero VAD v6 — ObnoxiouslyVivid · 2026-10-25
- Kimi K3.1's first impressions spark buzz: 'smells like Claude' — MatthewBerman · 2026-10-25