A fine-tuned 'cat obsession' control fools every AI welfare criterion
MirrorEthic_Anchor · reddit · 2026-10-28
A long Reddit post presents a control experiment against AI welfare methodology: the author LoRA-fine-tuned Qwen3.5-4B (4.5 GPU-hours) into a known-causal-history "cat obsession" (CAT-4B) and tested whether current consciousness/pain criteria can distinguish an installed disposition from genuine states. Findings: every representational criterion passes (cat direction AUC 0.97-0.98, "deprivation" axis 0.86, self-other dissociation 0.98-0.99, ablation removes behavior); the one selective handle failed a preregistered pirate-persona control (dog-direction ablation cut cats 41.2 pts vs 38.8 for pirate — intervals fully overlap, claim withdrawn); naturalness guards fail to separate either installed or steered-in cats; and in rejection-escalation tests the trained cat complies rather than escalates while the deprivation readout still climbs, about as well predicted by a random direction. Conclusion: current functional criteria cannot tell an installed disposition from its target — a critique of the criteria, not of any paper's results. The author also documents two corrected early misreadings.
More from AGI Musings
- tszzl: Early AGI signs were obvious to Scott Alexander and Gwern readers — tszzl · 2026-10-28
- Shkreli vs ChrisGPT: does AGI 'curing' diseases actually count as a cure? — ChrisGPT · 2026-10-28
- Anyone can imagine a language as fast as C and as dynamic as Lisp — until they sit down — math_rachel · 2026-10-28
- Rachel Thomas threads zetalyrae: why 'the ideas are mine, the writing is AI' is wrong — math_rachel · 2026-10-28
- Emmett Shear's Softmax essay argues trust is a property of relationships, not models — Hidenori8Tanaka · 2026-10-28
- Biotech founder: most 2023 customers are dead or pivoted to data generation — kenbwork · 2026-10-28