sentence-transformers v6.3 ships token-pooling-aware training, cuts doc vectors 84.7% with 97.3% score kept
tomaarsen · x · 2026-11-11
sentence-transformers maintainer tomaarsen released v6.3.0, headlined by token-pooling-aware training for multi-vector (late interaction) embedding models plus optional GPU kernel acceleration.
Key updates
- Token-pooling-aware training: a new MultiVectorTokenPoolingLoss trains on both full and pooled document representations so the encoder preserves matches after merging token vectors. It stacks with Matryoshka training to shrink both vector count and dimensions, plus quantization for storage.
- Kernel acceleration: optimized kernels for MaxSim scoring, Ward clustering pooling, and transformer backbones load from the HF Hub via sentence-transformers[kernels] and usekernels=True. The Ward kernel moves pooling onto GPU: 1.02s vs 7.00s on 5k SciFact docs (RTX 3090, BF16).
- Also: native AWS Neuron inference, faster community detection and Dataset inputs, and encoding/training/eval fixes.
Results: training two ModernBERT-base models on the same 120k MS MARCO triplets with only the loss changed, pooling-aware training hits NanoBEIR nDCG@10 of 51.23 at 49 vectors/doc (vs 193 unpooled), retaining 97.8% of its unpooled score vs 94.6% for standard training. Adaptive pooling at 30 vectors/doc keeps 97.3% of score with 84.7% fewer vectors. A trained checkpoint, kernels Hub pages, and a companion blog post with training code are available.
More from Infra
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12
- Potion UI's local login system turns your PC into a multi-user AI server — cocktailpeanut · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12
- AWS Finally Built Its Own AWS Emulator — super_linear · 2026-11-12
- Megakernel is multiprogramming for GPUs: 'GPUs are entering the CPU age' — pranjalssh · 2026-11-12
- Gemini Decision 1 costs $345 per 1M tasks, 3x pricier than leader Drex Max v1.1 — ArtificialAnlys · 2026-11-12