Praxist: From Experimental Artifacts to Solution Lineages
Jin Li, Ahmed Murtadha, Zhiyu Wang, Qiwen Chen, William Chen, Yifei Wu, Guan Wang, Andy L. Siy, Jiayi Yang, Mengsha Huang, Wenhao Li, Yixuan Liu, Shuailin Pan, Mingli Yuan, Sen Song, Yuhao Sun
cs.MA, cs.SE
2026-08-27
Praxist lets R&D agents inherit typed evidence across generations. On 75-task MLE-bench it takes 60 medals (49 gold) at $3,054 vs 55 medals and $38,370 for Claude Code on Opus 4.8.
Autonomous R&D agents already write code, run experiments, and revise against an external score. The open question is how a long campaign should remember. Most systems keep state as a search tree over candidate solutions: logs, memories, and scores record what happened, without saying which design element caused the gain, whether that evidence survived validation, or how it recombines with others. Long runs keep relearning the same lessons.
Assembly theory offers a different primitive. Complex objects are formation histories of reusable parts, and selection decides which parts persist. Praxist turns that into a systems requirement it calls evidence inheritance. A result is not passed down as a raw transcript or a scalar. Before it can shape later work, it gets an operational role: a mature parent, a fragile candidate, a failure to avoid, or a constraint to keep.
Praxist runs a generational cycle: artifact, finding, frontier, agenda, lineage. At the start of a generation, an allocator issues one design contract per parallel peer. The Deep Innovation Gate forces the peer to name the mechanism, the intervention surface, the parent lineage, the evidence signature, and the validation hook before any code is written. Quantified Diversity then spreads those contracts across distinct cells of mechanism family, intervention surface, and intent, so a generation tests several hypotheses instead of crowding the direction that currently looks best.
Peers do only local work. Each builds a reproducible artifact, an external evaluator owned by the task scores it, and the outcome is interpreted against the contract into typed findings: positive, negative, diagnostic, uncertain, or procedural. Failures are first-class evidence. Later generations inherit them as constraints rather than rediscovering them.
Global synthesis happens at the generation boundary. A PI panel reads the pooled findings from Builder, Skeptic, and Portfolio seats, with an External-validity seat in high-stakes mode. A Chair turns the memos and the updated frontier into the next agenda: continue, stop, validate, or explore. The frontier is split into confirmed, candidate, diagnostic, and validation lanes; immature smoke or scout scores cannot crowd out a complete result. Durable lessons can be compressed into Gems. That facility is off by default; among the four case studies only trading used it, every six generations, with at most four active Gems. A campaign reports two things: the best-scoring confirmed artifact, and the lineage ledgers that explain how it was reached.
The primary benchmark is the full 75-task MLE-bench suite, one local sweep, medals from the official grader. Praxist uses deepseek-v4-pro with a 1M-token context, up to 20 generations of 12 peers per task. The control is Claude Code on Claude Opus 4.8 with maximum thinking, same harness, same H100 pool.
| System | Any Medal | Gold/Silver/Bronze | Model spend |
| Claude Code + Opus 4.8 | 55/75 (73.3%) | 34/16/5 | US$38,370 |
| Praxist + deepseek-v4-pro | 60/75 (80.0%) | 49/10/1 | US$3,054 |
Any-medal by tier: Low 90.9% vs 81.8%, Medium 81.6% vs 76.3%, High 60.0% vs 53.3%. On the 70 tasks where both arms scored, the baseline has the better raw score on 36 and Praxist on 33, with one tie. The gap is medal grade, not a majority of head-to-head scores. An internal integrity pass rejected 90,423 contaminated attempts and substituted a clean actor on 9 tasks. Claude Code left 5 tasks unscored after a severe-cheating screen.
Four open-ended case studies reuse the same loop.
Rocket landing: on a frozen 6DoF plant, the selected deterministic controller lands 12,288/12,288 trajectories. The starting artifact scores 4.03%; Weco, an autonomous code optimizer on the same task and objective, reports 17.12%. Recorded spend is US$196 vs US$1,010. Weco could edit the task-variant manifest and used a different model, so the paper marks it as an external reference, not a matched comparison.
Quantitative trading: over 28 walk-forward quarters from 2019Q1 to 2025Q4, an LSTM-PPO policy compounds at 53.07% calendar CAGR against 22.80% for a paired equal-weight baseline. That artifact sits in the incubator lane (one seed, three hard constraint violations). The confirmed-lane champion is a cleaner attention policy at a lower CAGR. The headline number is a post-hoc pick on the reported metric, not the campaign's own promotion.
SLAM: CovSched, a visual-update scheduler on FAST-LIVO2, cuts evaluator-captured visual-path time by 72.4% on average across 14 NTU-VIRAL sequences. Mean APE RMSE reads 0.0937 m vs 0.0501 m, but the two arms stamp poses differently; re-associating them under one rule almost erases the gap. The paper claims no accuracy gain.
Tokamak magnetic control: versus a MAST-U PCS-style controller, aggregate survival is 1,264/1,500 vs 1,222, and common-horizon tracking error is 2.86 vs 2.99. On the original full-horizon metric the PCS-style controller is better, 4.42 vs 4.65. Neither clears the official pass line. Both closed-loop controllers read target errors from the harness, not from probe observations.
The practical claim is narrow. For long-horizon R&D agents, the bottleneck is often whether an evaluation becomes a recombinable mechanism, not whether the agent can write another patch. Praxist makes that interface a lineage over artifacts, and releases the code plus the generational ledgers.
The cost contrast is not "the same model, twelve times cheaper." Praxist sits on DeepSeek; the baseline sits on Opus 4.8. What still travels is that a cheaper model, with typed inheritance and a lane-structured frontier, cleared more medals and a higher gold share (81.7% vs 61.8%) than a much more expensive single-agent sweep. If a task already has an executable evaluator (Kaggle-style submit, controller sim, strategy backtest), this loop is closer to sustained engineering than another search tree.
On raw scores, the MLE-bench result is a threshold improvement, not a sweep of the leaderboard.
The paper draws its own fences. MLE-bench is one local sweep, not an official multi-seed leaderboard row; the margins are campaign outcomes under a stated protocol, not variance estimates. The rocket banks were adaptively reused, so selection-overfitting is on the table; scoring stops at first contact, uses exact state feedback, and a frozen low-order plant, and is not a real-world landing claim. The 53% CAGR trading policy never cleared promotion. The SLAM accuracy columns are dominated by a timestamp confound. Fusion is an internally matched comparison inside a privileged-state harness.
A missing control is the obvious one: Praxist versus a plain multi-try or tree-search loop on the same base model. Without it, medal gaps cannot be split from model, scheduling, and adjudication differences. Gems are off by default, and the default config also leaves the PI panel off. Lineage as a deliverable is documented in the four case studies; the 75-task sweep reports graded outcomes only.