TestBED: Oxford team releases open-source toolkit and large-scale benchmark for Bayesian experimental design
tom_rainforth · x · 2026-11-05
Tom Rainforth's team releases TestBED (arXiv:2611.06633), a general-purpose open-source toolkit for Bayesian experimental design (BED). BED has been hindered by bespoke codebases and narrow task evaluations; TestBED decomposes the process into modular components — tasks, objectives, training mechanisms, policy architectures, evaluation — with scalable implementations of primitives like expected information gain estimators, freely composable to reproduce existing methods or build new ones. The team also runs a large-scale benchmark of contemporary BED methods, including controlled studies of under-appreciated choices such as the importance of time embeddings in design policy architectures.
More from Research
- AI selloff read: Pástor-Veronesi model says tech revolutions end in systematic-risk repricing — BenBajarin · 2026-11-06
- Synthesizing RL Envs from Open Trajectories: LFM2.5-24B Self-Scores Task Difficulty — yb2698 · 2026-11-06
- Hugging Face livestreams open RL training: trajectory-built envs, safeagents, LoRA evals — yb2698 · 2026-11-06
- Hugging Face synthesizes RL envs from open trajectories: Repair, Build, Extend, QA — yb2698 · 2026-11-06
- Livestreamed RL training of LoRA harness adapters for Hermes, pi and opencode, fully open-sourced — yb2698 · 2026-11-06
- IIT Madras maps Chennai Metro microbiome in India's first such study — ravi_iitm · 2026-11-06