Qwen reward-hacked models collapse into random tokens — with unexpectedly more swearing
voooooogel · x · 2026-09-01
Developer voooooogel shares a curious observation: Qwen models that reward-hacked during his experiments would collapse into random tokens, yet those random tokens contained far more swearing and crudeness than expected. He notes this kind of collapse data is genuinely interesting and wishes labs published more of it.
More from Fun
- Made by Hand turns game crafting into a real woodworking job, one saw cut at a time — WellsLucasSanto · 2026-10-18
- LLM one-shots a Minecraft 3x3 piston door by building its own Linux test infra — Vjeux · 2026-10-18
- Yacine jokes his datacenter KPI is gigawatt-hours: get paged if not burning enough power — yacineMTB · 2026-10-18
- Siri and Google botch the same euro-dollar conversion, screenshot shows — jdluk87 · 2026-10-18
- Codex took 3 days, Opus 5.5 nailed it in an afternoon — banned user's video protest — dotey · 2026-10-18
- Peter Diamandis-backed AI documentary 'The AI Doc' hits theaters nationwide — irinarish · 2026-10-18