Fine-tuning Qwen3-1.7B on a 16GB GPU to write coherently without the letter 'e'
CatAstro_Piyush · x · 2026-10-26
Kevin Wu shared a hands-on RL fine-tuning experiment: training Qwen3-1.7B locally on a 16GB GPU to answer everyday prompts without ever using the letter 'e' (a lipogram constraint).
Key takeaways:
- The task is surprisingly hard for the same reason LLMs struggle counting r's in "strawberry": models read tokens, not characters, so they can't directly see whether text contains an 'e'.
- Simply masking out the 'e' tokens produces gibberish; RL training lets the model learn to write coherently around the constraint.
- The post includes before/after examples showing the fine-tuned model producing fluent, e-free poems and answers.
Full writeup and code are public, making it a reproducible intro-level RL fine-tuning project.
More from Research
- Titans learns during inference—but what if it learns the wrong thing? An experiment shows recall never fully recovers — pardhu-- · 2026-10-26
- Hugging Face publishes a five-step curriculum for post-training AI agents — lmoroney · 2026-10-26
- HAPAX paper: abstractive ICL emerges even without induction-head copying — davidbau · 2026-10-26
- HiP3R stitches local 3D reconstructions to keep long-video scenes consistent across thousands of frames — ccloy · 2026-10-26
- AlphaFold's dataset took 50 years to build — and cell simulation is far harder — shae_mcl · 2026-10-26
- ThousandML Open-Sources CQ 4-bit QLoRA: 17.4% Lower Val Loss Than Unsloth NF4 — textclf · 2026-10-26