Pairwise Bellman Validation: tuning-free model selection for offline RL and OPE
LarsvanderLaan3 · x · 2026-10-28
Researchers propose Pairwise Bellman Validation (PBV), a fast, tuning-free model selection method for offline RL and off-policy evaluation.
- Tests each candidate Q-function via pairwise critics Qi − Qj (and occupancy ratios via log(wi/wj)) — no auxiliary critics needed
- Requires no additional model fitting or hyperparameter tuning
- Comes with oracle guarantees even when every candidate is misspecified
- Selects via arg min max with a penalty term
A practical validation tool for OPE/offline RL experiments.
Related event: PBV Method for Offline RL Model Selection Draws Peer Scrutiny(3 posts)→