Multi-Budget Training Makes Retrieval Model Compression More General
antoine_chaffin · x · 2026-07-07
Instead of training for a single budget, the author trained the model with multiple compression budgets ([4,8,16,32,64,128,300]) at each training step. This multi-budget strategy makes the model easier to pool at any compression level. However, if the deployment target is known, training specifically for r=32 can retain 1.6% more information (reaching 97.4% compared to 95.8% for multi-budget) under the same λ, without sacrificing full token quality.
Related event: hpool Enables Flexible and Adaptive Compression for Retrieval Models(5 posts)→
More from Research
- NUS builds a soft force sensor that drives actuators without electronics or power — CurieuxExplorer · 2026-07-27
- Chelsea Finn says robot RL is bottlenecked by physical rollout cost, not algorithms — ycombinator · 2026-07-27
- ICML 2026 oral paper replication scores stay middling after a stricter re-scoring — profjamesevans · 2026-07-27
- Long-running agents will need immutable event logs, this thread argues — sebpaquet · 2026-07-27
- Seed IQ navigates Doom II, prompting questions about benchmarks beyond ARC-AGI — Fit_Transition8824 · 2026-07-27
- Agentic Data Science in Practice: Agents Write Code but Answer Wrong Questions — hugobowne · 2026-07-27