Study Traces Recommender's Last-Item Reliance to Residual Connections in Transformers
_reachsumit · x · 2026-08-17
Finding: Transformer-based causal self-attention recommenders (e.g., SASRec) often rely heavily on the most recent interaction at inference time, but the structural mechanism was unclear.
Conclusion: Research finds that while self-attention aggregates context, residual addition sharply shifts the representation toward the same-position contributions, termed "residual dominance."
Verification: Using inference-time residual scaling as an intervention, the study shows a monotonic trade-off between structural mixing and last-item reliance. Reducing residual strength recovers some hits where non-final positions already ranked the correct item.
More from Research
- Carmack critiques streaming RL: discarding observations is poor design — ID_AA_Carmack · 2026-08-17
- Agentic Science Pipeline Generates Novel Protein Sequences from Underspecified Goals — BrianHie · 2026-08-17
- Deep RL's "surrogate loss" has no physical meaning — author admits it's really a "fake loss" — burkov · 2026-08-17
- New Benchmark Reveals AI-Generated Video Detectors Fail on Real-World Crisis Events — huggingface · 2026-08-17
- HumanTracker Benchmark Released for Humanoid Motion Tracking Evaluation — GalaxyGeneralRobotics · 2026-08-17
- New Method Improves Reasoning Efficiency by Verifying Critical Claims — WeiboAI · 2026-08-17