Base models predict "her" via two separate pathways: token gender and post-verb pronoun rules
Sauers_ · x · 2026-10-02
An interpretability observation: a base model can correctly predict "her" after "The princess lost..." thanks to two independent computational pathways — the femaleness feature carried by the "princess" token, and a separate mechanism that predicts pronouns following verbs. Both converge on the same output.
More from Models
- SGLang ships Day-0 serving support for new model DaisyRose-27B — ying11231 · 2026-11-19
- Tibo's spicy take on why Astra 6.2 is getting such bad reviews — eigenrobot · 2026-11-19
- User builds interactive 3D molecular viewer macOS app with GPT-6.2 Astra — DeryaTR_ · 2026-11-19
- Caliper Harness: open-source coding harness disciplines small local models with plans, checks, reviews — Saraozte01 · 2026-11-19
- DaisyRose-27B's in-house sandbox: ~80% cheaper than cloud providers, 99.95% success rate — BanghuaZ · 2026-11-19
- Tencent's 220B Hy4-flash quantized to 49 GB weights, runnable on 64 GB VRAM — gladkos · 2026-11-19