Sentence Transformers packed position IDs wrong for RoBERTa models under flash attention
tomaarsen · x · 2026-07-23
Since v5.5.0, Sentence Transformers has packed text-only batches into a single sequence when flashattention2 is enabled.
The bug was that packed positionids restarted at 0 for each text, but RoBERTa-family models expect paddingidx + 1. That shifted every token's position embedding, causing silent quality degradation rather than a crash.
Related event: Flash Attention Causes Performance Drops in BGE-M3 and Other Models(2 posts)→
More from Research
- Stanford HAI says Evo 2 underpins the largest open language model and AI-made genomes — StanfordHAI · 2026-07-23
- MagNET predicts NMR in seconds as an ML surrogate for DFT calculations — CatAstro_Piyush · 2026-07-23
- Domenic Denicola shares his July 2026 agentic coding setup — cnakazawa · 2026-07-23
- A new notebook shows learning-rate boundaries can be fractal — S_Conradi · 2026-07-23
- Production agentic systems need a context layer, not just LLM tool calls — Pavan_Belagatti · 2026-07-23
- Nature asks whether organoid intelligence could become the next computing paradigm — LadiesMan-8 · 2026-07-23