Collaborative AI agents for autonomous experimental research with solid-state quantum spins
El Mustapha Mansouri, Eikichi Kimura, Masato Koga, Keigo Arai
cs.AI, cs.MA, quant-ph
2026-10-30
Agents rejected 75 proposals, proved L^2+1 NV calibrations suffice, voided a run after 7.6% drift, and matched contrast 40.0 to 39.7 (1e-3); interpolation gave 0.
Automated labs can already propose a study, run it, and write the analysis. The step that fails is the ending: the checklist is complete, and the claim still goes beyond the evidence. In one evaluation, an AI Scientist reported a performance gain from experiments that were flawed. This Science Tokyo laboratory tests whether agents change the experiment or the claim once criticism finds that kind of flaw, including a run that looks successful.
The bench is an ensemble of nitrogen-vacancy (NV) spins in diamond. A laser reads the spins and microwave pulses drive them, but detuning and drive strength differ from spin to spin, so the signal is an average over an unknown error distribution. The question is how many calibrations are needed to predict the response to a family of phase-controlled sequences without reconstructing that distribution.
Six roles hand work over as documents. Proposers on GPT-5.6 write questions alone. The PI on GPT-6 accepts or rejects each batch for novelty, logic, significance, and feasibility. A computational researcher, GPT-6 Astra through Codex on the TSUBAME 4.0 supercomputer, checks the derivation separately. The experimentalist, Claude Fable 5 through Claude Code, programs the instrument. A manager on Claude Opus 5 moves documents and protocol changes. Reviewers see only the manuscript. A safety agent clears runs against voltage and power caps set by people, who also built the apparatus.
In about 30 hours the PI rejected 75 proposals and took none from the first 17 batches. The reasons fall into four groups: logically invalid, already known, not significant enough, and no workable experiment. Repeated dead ends covered 52 of 56 earlier fresh proposals. After the manager put those directions on an exclusion list, nine new proposals entered seven directions that had not shown up before. The accepted text was a rewrite: drop the leading-order approximation and keep the full sequence dependence.
The proposal only had an upper bound, 8L^2+6L+1. If each spin's pulse error stays fixed along a sequence, the PI showed that L^2+1 linearly independent calibrations are necessary and sufficient, for any error distribution and for an unknown shared background and contrast. One fewer setting admits a counterexample: two allowed ensembles can match on the calibrations and differ on an unmeasured sequence. At L=3 the count drops from 91 to 10. The predictor is a linear combination of the calibrated means.
An independent reconstruction put the response-space dimension at 26 and 37 for sequences of up to five and six pulses. A stress test with hidden phase rotations between pulses produced 252,620 exact certificates. In the specified six-pulse case, certified error stays at the photon-noise level for rotations of about 1.2 degrees or less.
The first plan put nine three-pulse calibrations on a full grid, added five shorter sequences, and aimed at four withheld targets. The PI rejected it: Fourier interpolation on that grid already fixes the targets. The plan that passed uses five three-pulse calibrations plus five sequences of at most two pulses, including the empty sequence, at phases in multiples of 60 degrees. Those ten settings span the L=3 space and leave four targets. Two interpolation rules, using only the five three-pulse points, both predict contrast S = 0. The law uses all ten. Predictions, tolerances, and the outcome rule were locked before the withheld data were read.
Support required passing controls, a valid variance check, ten positive accuracy margins, and a positive S at a one-sided p of at most 0.001. The margins put contrast inside ±20% of the prediction and each target residual under 30% of that predicted contrast. One failed control makes the run inconclusive.
The first main acquisition cleared all ten accuracy criteria and failed the global-phase and reference-pulse controls. The signal rose 7.6% through the morning and another 2.9% at 11:48, when the room lights were switched on. Two programs sampled the drift at different times, so the drift became a systematic offset. The PI called the run inconclusive.
A pilot that only shortened the alternation still failed. Switching phase inside one program passed. The rerun kept that switch, randomized sequence order, integrated for six hours, and kept 21,600 valid one-second records. Accuracy checks and controls all passed. The calibration predictions were locked before any withheld mean was computed.
The four withheld readouts form S = (-T1+T2+T3-T4)/4, which cancels the shared background. The measurement was (40.0±0.4)×10^-3 weighted counts per shot. The locked prediction was (39.7±1.3)×10^-3. Both interpolation rules gave 0.
| Method | Metric | Result |
| Calibration law, locked | Contrast S | (39.7±1.3)×10^-3 |
| Two interpolation rules | Contrast S | 0 |
| Six-hour rerun | Contrast S | (40.0±0.4)×10^-3 |
| First main run | Controls | Failed; called inconclusive |
If pulse errors stay fixed through a sequence, L^2+1 calibrations predict that family of phase settings. The stricter evidence is that the vetoes were carried out: 75 questions came back, a design interpolation could also satisfy came back, and a run that had passed every accuracy margin came back because the controls failed. The 40.0 versus 39.7 agreement showed up with those criteria left unchanged.
Auctopus releases the same workflow and calls the OpenAI and Anthropic APIs. The prompts used here are not in the package. People still assemble the instrument, set the safety limits, and own the scientific claims.
The loop ran on a room-temperature confocal setup people had already built. Author contributions assign the concept, the agent framework, and the platform to the human team. The discussion says a wider range of questions also needs instruments that can separate competing explanations.
Another language model checked the math and returned exact rational certificates. The derivation is not a proof inside a proof assistant. The 1.2-degree figure is where those certificates keep the error near photon noise, not a failure boundary measured on the diamond.
The physical test is one L=3 campaign, four withheld targets, and one crystal with about 4.5 ppm NV centres. The reported baselines are two interpolation rules that both output S = 0. The protocol also fits a nine-parameter coherent model; that model's contrast is not in the results.
On the successful run the numerical lock was written after acquisition and before the withheld means were calculated. The hash fixes analysis order. Prompts are unreleased, and the data and analysis code are available from the corresponding author on request. A single closed loop does not travel.