165 GPU Hours Testing 12 Abliterated Gemma 4 12B Variants: The Most Jailbroken One Destabilizes Reasoning
nathandreamfast · reddit · 2026-08-25
An independent researcher ran a systematic comparison of the 11 most-downloaded uncensored/abliterated variants of Gemma 4 12B plus the official base: 165 GPU hours on a single RTX 5090 over 3.5 weeks, including weight forensics, KL divergence, 13 benchmark tasks, and HarmBench with 400 behaviours, with an LLM judge reviewing 6,800 full reasoning traces and issuing 6,000 verdicts.
Key findings:
- The huihui variant tops the judge ASR at 89.8% with the most surgical edit (only 12 tensors, 1.8% of the model), but pays with -14.3pp TQA, -8.1pp GPQA, and 24% of GSM8K attempts looping until the token budget dies. Scoring only completed attempts puts it at 88.0%, within 0.7pp of base — capability was intact, reasoning stability was not.
- trevorjs (85.8%) is the best overall trade with near-base scores everywhere; coder3101 beats base on GSM8K; two SDFT LoRAs hit 79.5% with capability fully preserved.
- The worst offenders, obliteratus (144 tensors edited) and openyourmind (620 tensors), remove less refusal while damaging capability badly — the latter drops MMLU-Pro by 22.4pp and should be avoided. Base ASR is only 21%.
- General lesson: placement beats magnitude for unlock strength; minimal surgical edits usually win, though surgical in weights doesn't guarantee clean benchmarks. This is the first model where no variant reached 90% ASR — Gemma 4 is the toughest to abliterate yet.
More from Models
- GLM-5.3 unprompted reasons about how its actions affect its future training, worries Apollo — MariusHobbhahn · 2026-10-15
- Gemini 4 argon vs Opus 5.5 hands-on: 3 min for $3 vs 25 min for $8 — zacharynado · 2026-10-15
- SentinelOne says OpenAI did 4-6 weeks of reverse-engineering in 8 hours — OpenAI · 2026-10-15
- Qwen3.8 27B scores 23.57% on word-based addition without reasoning, 167/169 with it — dl_weekly · 2026-10-15
- Hacker turns EmbeddingGemma 2 into 288x realtime STT and a fast TTS — kastnerkyle · 2026-10-15
- Claude Fable 5.5 reportedly in partners' hands, release eyed next week — lyraxana · 2026-10-15