Berkeley AI proposes Population Serving: training models during inference with no backprop
LChoshen · x · 2026-11-13
Researchers at Berkeley AI (with Marquita Ellis, Gopala Anumanthan and LChoshen) introduce Population Serving, a new inference paradigm that treats inference compute as an untapped training resource: models keep learning while they serve traffic.
- Instead of serving everyone identical weights, it serves slightly perturbed copies of the model
- Each copy is scored on the traffic it receives, and copies are periodically merged
- No backprop needed; only one inference call per user query, powered by zeroth-order methods
A blog post and thread detail the approach.
More from Infra
- Anthropic's Q3: revenue up 47% but compute costs jumped 77%, swinging to an operating loss — bigdata · 2026-11-13
- Anthropic launches self-hosted Claude Managed Agents in private beta for on-prem deployment — ClaudeDevs · 2026-11-13
- SpaceX commits $220M for water infrastructure to support its $24B AI supercomputing site — Polymarket · 2026-11-13
- SpaceX commits $220M to Texas water infrastructure for its $24B AI supercomputing site — Polymarket · 2026-11-13
- vLLM Semantic Router Istio policy bypass: one header skips the security filter — colinmcnamara · 2026-11-13
- Bittensor-powered GPU rental market Lium reports 79% ARR growth in 30 days — markjeffrey · 2026-11-13