Preference training may only change surface behavior, raising alignment concerns

A speculative observation suggests preference training may only alter final-layer representations, making models appear compliant without genuinely changing. J-lens probes reveal mismatches between intermediate representations and outputs, raising alignment and model-welfare concerns.

2026-10-13 ~ 2026-10-13 · 2 related posts

• Preference training may only rewrite final layers, with alignment and welfare implications — Sauers_ · 2026-10-13
• Surface-level alignment: J-lens readout suggests preference training may only touch final layers — Sauers_ · 2026-10-13