2026-08-26
Orbformer, pretrained on 22,350 structures, is the only method here that consistently reaches 1 kcal/mol on Diels–Alder and five bond-breaking paths.
Bond breaking is where quantum chemistry gets both expensive and unreliable. Near equilibrium geometries, the electronic wavefunction is usually weakly correlated, and single-reference methods such as coupled cluster can be extremely accurate. Stretch the bond and the wavefunction becomes strongly correlated. There is no default multireference method. Active spaces need expert tuning per system, and the cost scaling is often worse than on the single-reference side.
Standard ab initio calculations also restart from scratch on every molecule, even when two systems share the same local chemistry. Recurring electronic-structure patterns get paid for again and again. Deep quantum Monte Carlo (deep QMC) represents the wavefunction with a neural net and trains by energy minimization, which makes it unusually well placed to store those patterns in one set of weights. Earlier transferable attempts pretrained on at most about 700 structures. That is below the scale needed for a practical tool.
Orbformer takes the molecular configuration M (nuclear positions and charges) as a network input, not as a frozen parameter of a one-off optimization. One set of weights can then describe molecules of different size, composition, and geometry. The wavefunction is a Jastrow factor times a sum of generalized Slater determinants. Electron features run through an Electron Transformer; orbitals are produced on the fly by an Orbital Generator that sees the nuclear environment. Locality is the design rule: pairwise interactions decay with distance, and orbitals are constrained to be localized so similar chemical environments can reuse them.
Training generalizes variational Monte Carlo. An outer loop samples molecules from a training distribution; an inner loop samples electron coordinates from the current |Ψ|²; the objective is the expected energy. No labeled energies are required. The electron data is generated by the model itself.
The pretraining set is the Light Atom Curriculum (LAC): 22,350 configurations, up to 24 electrons, atoms limited to H, Li, B, C, N, O, and F, with bent, stretched, and dissociating geometries baked in. CCSD(T) T1, D1, and |%TAE| diagnostics flag about 20% to 45% of structures as multireference. Pretraining runs in three stages: bent and stretched geometries with at most 10 electrons (200k steps, about 320 A100-hours), then all geometries in that size band (another 320 hours), then the full LAC (400k steps, about 4,000 A100-hours).
Chemical accuracy of 1 kcal/mol still needs fine-tuning from those weights. Target structures are grouped, typically a whole path that shares the same nuclei, and trained with one parameter set. Separate single-point optimizations are the expensive baseline this is meant to beat.
A few training changes make the scale feasible. Electron samples for unseen molecules are burned in with Unadjusted Langevin, then switched to MALA with accept/reject to match the target distribution. Hartree–Fock initialization is skipped because it is unreliable in the multireference regime. A penalty keeps multiple determinants from collapsing so that one term does not dominate early and freeze the model into an effective single determinant. The net trains stably in TF32, with electron batches at least 4× smaller than Psiformer. FlashAttention and a custom Forward Laplacian kernel cut the remaining wall time.
The concerted Diels–Alder transition state of ethene plus butadiene has 46 electrons, well outside the pretraining distribution. Orbformer’s activation energy sits essentially on the experimental value; the reaction energy also lands within 1 kcal/mol of experiment. The stepwise biradical path comes out about 9.9 kcal/mol above the measured activation energy, and the concerted path is about 10 kcal/mol below the stepwise one, matching current experimental and theoretical consensus. Among classical wavefunction methods, only multireference coupled cluster matches experiment, and only with a small basis or an artificially restricted configuration space.
Five bond-breaking minimum-energy paths cover ethane, formamide, 1-propanol, 2-aminopropan-2-ol, and L-alanine, 20 points each, up to 48 electrons. No experimental surfaces exist, so the reference is expensive single-point deep QMC. On mean absolute relative energy error, LAC-finetuned Orbformer sits on or ahead of the Pareto front formed by DFT and NEVPT2/MRCI/MRCC. Error falls monotonically with compute. Classical methods often cannot be converged in a similarly systematic way.
| Setting | Cost vs training from scratch |
| Joint path fine-tune vs single-point | about 20× cheaper |
| Ethane (inside LAC) to 1 kcal/mol | about 16× |
| 1-propanol (slightly larger than LAC) to 1 kcal/mol | about 6× |
| L-alanine to 5 kcal/mol | about 8× |
| L-alanine to 1 kcal/mol | only about 10% faster |
Relative to the best prior single-point neural ansatz, cost to chemical accuracy drops by about two orders of magnitude. On TinyMol, pretrained Orbformer is best in- and out-of-distribution, and reaches 1 kcal/mol at least an order of magnitude faster than training from scratch. Gao et al.’s transferable model is hurt by more pretraining on out-of-distribution molecules; Orbformer still benefits.
The model also learns, without being told, to place exactly two fully localized core-like orbitals on each carbon. Slater matrices for octane and heptane are nearly identical at the end whose local environment matches, which is direct evidence that orbitals are reused across molecules. Alkane fine-tuning reaches 106 electrons.
This is the first time amortizing the cost of solving the Schrödinger equation across many molecules looks like a practical tool, rather than another deep QMC accuracy demo. Code and weights are open. For organic bond breaking, transition states, and multireference reaction paths, it is a credible high-accuracy reference, especially when active-space choice and basis-set sensitivity are the headache.
It is not a zero-shot foundation model. Chemical accuracy requires fine-tuning, and the public checkpoint covers only those seven light atoms. The intended large-scale use is as a data generator for cheaper reactive force fields.
For people building neural wavefunctions, the portable pieces are concrete: ULA/MALA burn-in, a determinant-balance penalty, and locality plus composability constraints. Those travel better than another attention layer.
The authors themselves exclude ions, spin assignments other than Sz = 0 or 1/2, and excited states. The pretrained element set is closed. Composability does not give size consistency once multiple determinants are used. Far from the pretraining distribution, the pretraining bonus nearly vanishes after long fine-tuning: L-alanine is only 10% faster to 1 kcal/mol.
On the dissociation paths, “chemical accuracy” is measured against a more expensive deep QMC reference, not experiment. Using a cousin of the same variational method as the gold standard is circular. Diffusion Monte Carlo is not in the comparison set. Diels–Alder is a harder experimental test, but it is one reaction.
“The only method that consistently reaches 1 kcal/mol” holds for the baselines in this paper. Transition metals are untested. Large conjugated systems have no dedicated experiment. Public weights are not meant for zero-shot use.