Masked decoding trick tames text rivers in Gemma 3 12B, with a width-aware LoRA and coherence RL

cephaloform · x · 2026-11-14

Developer erinbeess shared a technique for controlling text layout in Gemma 3 12B generations: masked decoding at inference time, where any token that doesn't match the desired length gets masked out, eliminating unsightly 'text rivers' in generated paragraphs.

The full pipeline:

Token-length-based masking during decoding to filter out ill-fitting tokens
A LoRA trained with width tags so the model becomes aware of text length
Coherence-oriented RL fine-tuning afterwards

The accompanying screenshot shows even, well-aligned lines without the repetitive stacking typical of small models. A hands-on decoding-control trick worth replicating.

Original post →

Masked decoding trick tames text rivers in Gemma 3 12B, with a width-aware LoRA and coherence RL

More from Models