Score Centering Is Infra-Unfriendly at Long Context: Two Huge [T, K] Tensors and a Custom Kernel

dejavucoder · x · 2026-11-20

dejavucoder quote-tweeted a technical remark noting that score centering is not very infra-friendly under long context: it requires storing two huge [T, K] tensors and doing element-wise multiplication, plus a custom kernel. A short engineering note on attention/scoring implementation details.

Original post →

More from Infra

Infra channel →