CDRL: Certification-Driven Reinforcement Learning for Neutrino Flavor Model Discovery
Piyush Jha, Jake Rudolph, Victoria Knapp-Pérez, Max Fieg, Aishik Ghosh, Vijay Ganesh
cs.AI, cs.LG, cs.LO, hep-ph
2026-08-21
CDRL turns failure certificates into SAT clauses, hitting 1.95× AMBer valid-model rate and 6.33× neutrino-model rate in a 10^26 space, with up to 4× fewer evaluations.
A neutrino flavor model is a discrete assignment of particle representations, Abelian charges, and flavon vacuum expectation values, then a Lagrangian-to-mass-matrix pipeline and a χ² fit to NuFit 5.3. The space exceeds 10^26 candidates. Each evaluation is expensive. Exhaustive search is not an option.
AMBer was the previous best automated search: a deep net plus PPO, driven by a scalar reward. A scalar reward changes preferences. It does not change which regions remain reachable. The same invalid construction can be proposed again, and the agent never learns which assignment caused the failure. CDRL's job is to turn that failure into a reusable ban.
Models are filled in blocks: pick the active ℤN, then representation and charge for each particle, then a vacuum expectation value for each flavon. A policy-value network proposes the next block. Monte Carlo tree search (MCTS) walks the tree. The physics pipeline scores completed models.
The extra piece is a symbolic referee. Domain rules are compiled once into CNF: exactly one representation and one charge per particle, exactly one vacuum expectation value per flavon. During search, Boolean constraint propagation (BCP, the unit-propagation step inside SAT solvers) maintains the partial assignment and drops already-unsatisfiable moves from the legal-action mask.
When a completed model fails, a certificate analyzer names the responsible subset and writes a conflict clause into a shared database. Three certificate types:
Workers share the clause database and the replay buffer, so one certificate is enforced everywhere. Physics evaluation is about 98% of compute; pruning exists to skip those calls.
After training, shallow decision trees on high-confidence MCTS states yield 40 readable rules. Those rules re-enter a later run as a soft bias: the policy is reweighted as P(a|s)(1+β ra(s)) with β=0.5. Hard constraints still win.
Same physics pipeline as AMBer, same three theory spaces. A neutrino model is a structurally valid candidate with χ²≤10 and at most 7 free parameters.
| Space | Method | Valid % | Neutrino % | Min params |
| A4×ℤ4 | AMBer | 19.53 | 0.02 | 5 |
| A4×ℤ4 | CDRL | 28.1 | 0.10 | 5 |
| A4×ℤN | AMBer | 13.79 | 0.03 | 5 |
| A4×ℤN | CDRL | 26.9 | 0.19 | 5 |
| T19×ℤ4 | AMBer | 11.02 | 0.03 | 4 |
| T19×ℤ4 | CDRL | 18.2 | 0.09 | 4 |
Sample efficiency is the sharper contrast. On A4×ℤN, AMBer evaluates 4 million candidates and finds 1,394 neutrino models; CDRL evaluates 1 million and finds 2,343. On T19×ℤ4, AMBer uses 24 million evaluations for 6,439 finds; CDRL uses 6 million for 7,019. GPT-5.3 few-shot sits near random search, about 0.2% valid and 0.00% neutrino models.
Ablations on A4×ℤN: drop the network or MCTS and neutrino discovery falls to near zero. Drop certificates and valid rate falls from 26.9% to 16.2%, neutrino rate from 0.19% to 0.03%, matching the AlphaZero-style baseline. Smaller rule-guided runs add about 1.5–2× on valid rate and 2–3× on neutrino discovery. Those absolute numbers are not comparable to the full-scale table.
CDRL's ℤN coverage is broader than AMBer's. AMBer's reward explicitly prefers higher-order groups; CDRL has no such term. The spread comes from pruning equivalence classes uniformly.
For combinatorial scientific search, this is a clean coupling of SAT conflict learning to an AlphaZero loop. Scalar rewards change preferences; certificates change the reachable set. When verification is 98% of the budget, cutting evaluations by 4× is worth more than a slightly better net.
The intended setting is a discrete design space, an expensive verifier, and an external tool that can name the failing assignments. Chemistry, materials, and automated theorem proving are listed as cousins. The current implementation is wired to this neutrino flavor stack. It is not a drop-in agent.
Some of the 40 extracted rules look like habits a model builder already uses, such as aligning charged-lepton-singlet charges or coordinating L/N/φ triplets. Others look like artifacts of the search trace. The authors have not asked domain experts to validate them independently.
CDRL needs a verifier that emits useful certificates. Without that, it falls back to ordinary RL. Certificate analysis and clause management could become a new bottleneck in larger spaces; here they sit under the 98% physics cost.
Compute is not matched: CDRL uses 32 CPU cores, AMBer uses 250 parallel environments, so the paper compares evaluation counts rather than wall time. Fair, and it also means a wall-time win has not been measured.
The best neutrino-model rate is still 0.19%. The 6.33× over AMBer is a ratio of two tiny numbers. Whether a chosen vacuum expectation value can actually arise from a potential is unsolved on both sides; both inherit AMBer's assumption.
The rule-injection experiments are smaller-scale and should not be read as another full-scale doubling. Some trees only cover charge values that happened to appear in high-confidence samples; the authors treat that as possible sampling bias, not a physical prohibition.