Base model predicts "her" via two separate pathways: token semantics and verb-based pronoun rules
Sauers_ · x · 2026-10-02
An interpretability observation: a base model can predict "her" after "The princess lost..." through two independent computational pathways — the femaleness encoded in the "princess" token, and a separate syntactic mechanism predicting a pronoun after a verb. The finding suggests redundant, parallel computations inside the model, where the same prediction is supported by distinct mechanisms combining semantic and syntactic cues.
More from Research
- LOCI: hybrid spatial linear memory lets streaming world models recall revisited scenes at ~30% less memory — IFM · 2026-10-02
- SAKIKO auditing shows +55 net-gain interventions corrupt over half of correct tool-using LLM decisions — UniversityofBirmingham · 2026-10-02
- Researchers warn AI-written 'salami' papers are flooding arXiv with low-value work — EhudReiter · 2026-10-02
- Arena Physica explains why FEM solvers never compute E-fields at mesh nodes — burny_tech · 2026-10-02
- AWSM grounds LLM-agent 3D scene reconstruction in IMU, depth and pose evidence — anselm · 2026-10-02
- Study links attention nonlinearity to power-law massive activations and scaling laws — burny_tech · 2026-10-02