Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders
Keito Kozaki, Keigo Sakurai, Ren Togo, Takahiro Ogawa, Miki Haseyama
cs.AI, cs.IR
2026-08-14
Hokkaido University traces SASRec-style recommenders' extreme reliance on the most recent item not to attention weights but to residual connections structurally preserving same-position information.
SASRec-style sequential recommenders, built on causal self-attention, tend to lean heavily on whatever item the user interacted with most recently. Prior work already confirmed this with perturbation experiments and ranking diagnostics, but never explained how it actually forms inside the model. If attention weights alone were driving it, influence should decay smoothly from the final position backward. What's actually observed is a cliff: keep only the last item and shuffle everything before it, and performance barely changes; shuffle the last item too, and performance collapses. That discontinuity means attention weights alone can't be the full story.
Instead of stopping at attention weights, the paper decomposes the entire attention block (self-attention plus residual connection plus layer normalization) using norm-based decomposition, which splits each position's contribution to the final representation into measurable components. This yields a mixing ratio: low values mean a position's output is dominated by self-information retention, high values mean it's aggregating more from other positions via attention.
The analysis proceeds in three stages, measuring the mixing ratio at the raw attention output, after adding the residual connection, and after layer normalization. It's run across nine public datasets spanning different sparsity levels, sequence lengths, and domains, and across SASRec, DuoRec, BERT4Rec, and GRU4Rec, to check the pattern isn't an artifact of one dataset or architecture.
To test whether the link between residual connections and last-item reliance is more than correlational, the paper adds a controlled inference-time intervention: residual scaling. A coefficient alpha (alpha=1 is the standard setting) directly modulates residual-branch strength at inference, without retraining, letting the authors check whether mixing ratio and last-item reliance move together and reverse as alpha changes.
| Metric | Value |
| Mixing ratio, attention output alone (avg across 9 datasets) | 0.72-0.96 (high, confirming attention does aggregate context) |
| Mixing ratio after adding residual | Drops sharply to 0.12-0.40 (self-preservation dominates) |
| Extra effect of layer normalization | Minimal, nearly identical to post-residual values |
| Position-wise variation (L, L-1, L-2) on Toys/ML-1M | Max difference 0.0032-0.0186; residual dominance is broad, not concentrated at the final position |
The residual-scaling experiment confirms the structural account: lowering alpha consistently raises the mixing ratio and lowers HRLI@1 (the rate at which the last item gets ranked first), moving in lockstep. Dropping alpha to roughly 0.1-0.3 also recovers a meaningful share of cases where the final-position prediction had missed the ground-truth item but an earlier position's representation already ranked it correctly within the top 10, indicating the correct signal was present inside the model all along, just structurally suppressed by the residual's preference for the last position. A validation-based alpha selection procedure reproduces the same accuracy-recovery trade-off seen with test-set oracle tuning, ruling out the finding being an artifact of test-set overfitting.
For teams building sequential recommenders, this points to an overlooked debugging angle: excessive reliance on the most recent interaction may not stem from the training data or loss function at all, but from an entirely ordinary architectural component, the residual connection. Adjusting residual strength at inference is a knob worth trying if you want to dial down sensitivity to the latest click, without retraining, though the paper is explicit that this is a diagnostic tool, not a deployable new method. More broadly, the lesson that attention weights don't tell you what information is actually used applies to any causal self-attention sequence model, not just recommenders, and is worth keeping in mind for interpretability work elsewhere.
The authors are explicit that residual dominance isn't the sole cause of last-item reliance; causal masking, positional encoding, the prediction interface, training objectives, and data characteristics could all contribute, and this paper isolates one mechanism rather than the full causal chain. The analysis and intervention only apply to SASRec-style causal models that predict directly from the final-position representation; the paper explicitly says the same framework doesn't transfer to bidirectional models like BERT4Rec, since BERT4Rec predicts from a [MASK] position whose residual input is a placeholder rather than an actual interacted item, so the mixing ratio's interpretation breaks down there. The relationship between diagnostics like HRLI and recovery rate and real online utility, such as user satisfaction or retention, is also flagged as an open question the paper doesn't answer.