Base models predict "her" via two separate pathways: token gender and post-verb pronoun rules
Sauers_ · x · 2026-10-02
An interpretability observation: a base model can correctly predict "her" after "The princess lost..." thanks to two independent computational pathways — the femaleness feature carried by the "princess" token, and a separate mechanism that predicts pronouns following verbs. Both converge on the same output.
More from Models
- Latest AI models are so good at reading binaries that 'every game will be effectively open source' — bradneuberg · 2026-10-02
- Ajeya Cotra's 2026 AI forecast review: 'plan a birthday party' has already fallen — ajeya_cotra · 2026-10-02
- Cantina releases apex-flash-1, an open-weights security model post-trained on real paid vulnerabilities — ctjlewis · 2026-10-02
- Unverified uncensored GLM-5.3 EXL3 3.0bpw quantized upload appears on Hugging Face — kevinnbass · 2026-10-02
- GPT-6.1 Sol and Claude Sonnet 5.5 Roll Out to Copilot, ChatGPT Debut Hinted — koltregaskes · 2026-10-02
- webAI ships 3.66B formal-logic model TwIL-LM3-Pro, 2.09 GiB and runs locally — kimmonismus · 2026-10-02