Sentence Transformers 5.6.1 fixes a silent flash-attention regression in RoBERTa models
tomaarsen · x · 2026-07-23
Sentence Transformers v5.6.1 fixes a silent flash-attention regression that affected RoBERTa/XLM-R-family checkpoints on the packed text-only path.
- The bug appeared since v5.5.0: packed batches with flashattention2 restarted positionids at 0 for each text, while RoBERTa-family models expected paddingidx + 1.
- That produced shifted position embeddings with no crash, just worse outputs.
- Impact example: BAAI/bge-m3 dropped from 0.8485 → 0.7239 on STSB Spearman and 0.6041 → 0.5414 on NanoBEIR mean nDCG@10.
- The patch restores quality exactly, while keeping the packing speedup.
Related event: Sentence Transformers 5.6.1 Fixes Flash Attention Embedding Regression(3 posts)→
More from coding & agent
- Claude becomes “Atlas” and manages a 6-agent fleet with two free MCP tools — Han_Thot_Terse · 2026-07-23
- Greptile says its mission is to automate code validation — garrytan · 2026-07-23
- Anthropic says Claude now writes 80% of its code, guarded by human approval gates — rseroter · 2026-07-23
- Domenic Denicola shares his July 2026 agentic coding setup — cnakazawa · 2026-07-23
- Production agentic systems need a context layer, not just LLM tool calls — Pavan_Belagatti · 2026-07-23
- ChatGPT Sites can build and deploy full-stack apps inside the chat — reach_vb · 2026-07-23