Tokenization refactor speeds up Sentence Transformers, biggest gains for static embedding models
tomaarsen · x · 2026-10-15
Sentence Transformers maintainer tomaarsen highlights Simon Brandeis' tokenization refactor, which meaningfully speeds up inference for some of the most popular Sentence Transformer models, with static embedding models benefiting the most.
The thread links to an in-browser demo measuring tokenization speed before and after the refactor, plus HF's earlier work on static embeddings that run 100-400x faster on CPU than models like all-mpnet-base-v2 while keeping 85%+ quality — enabling on-device and in-browser use.
More from Models
- Fireworks AI CEO: 0.1% per-turn numerical drift compounds to ~10% over 100 agent turns — AccBalanced · 2026-10-15
- kalomaze argues ensembled synthetic data can cover data-scarce domains — kalomaze · 2026-10-15
- Newer image models are easier to jailbreak, with 2x more attacks getting through — Scobleizer · 2026-10-15
- Gemini controls a real robot from one prompt: SSH in, find joints, learn from demo — heiga_zen · 2026-10-15
- Hugging Face tokenizers: another ~5x speedup for BERT tokenizers on top of v1 — tomaarsen · 2026-10-15
- Open-weights TYGER model combines ASR with full-duplex turn control, 250ms latency — KokaOP · 2026-10-15