How Molecular Generative Models Organize Molecular Identity
Raul Ortega-Ochoa, Tejs Vegge, Jens S. Bakander, Luis Mantilla Calderon, Alan Aspuru-Guzik, Tonio Buonassisi
cs.LG, physics.chem-ph
2026-08-07
Defining molecular identity as an equivalence relation and pulling it back through three generators reveals piecewise-constant chemical regions that subdivide coarse-to-fine; cohesiveness stabilizes before granularity in training, so latent navigability must be verified, not assumed.
Molecular generative models (variational autoencoders, diffusion models, autoregressive transformers) are used for drug and materials discovery. A common practice is to treat their latent space as a "chemical map": interpolate between two points expecting the molecule to morph smoothly from one into the other; walk along an axis expecting to optimize some property. This works only if the latent geometry corresponds smoothly to molecular identity. Does it? No one had directly characterized how these models internally arrange distinct molecules. This paper makes molecular identity explicit and pulls it back through the generative process to map the model's internal territory.
The key move is to define "what counts as the same molecule" as an explicit equivalence relation, in six grades from finest to coarsest: canonical SMILES, the first 14 characters of InChIKey, Murcko scaffold, Murcko-generic scaffold, molecular formula, and element composition. This identity is then pulled back through the generator: many points are sampled in latent space, each with the decoder's randomness exposed (the "random tape" eta), decoded into molecules, and each molecule labeled by identity. The result exposes the model's "repertoire", a fixed partition that determines which molecules it can produce.
Three architectures: MolMiner (a property-conditioned autoregressive transformer), HierVAE (a hierarchical graph VAE), and GDSS (a score-based graph diffusion model). Sampling is done on 2D sections; chemical cohesiveness of a neighborhood is measured by AUC(W,A), the probability that a within-neighborhood pair is more similar than an across-neighborhood pair (similarity via Tanimoto on ECFP fingerprints), with 0.5 being chance.
| Model | AUC(W,A) | Within-neighborhood median Tanimoto | Across-neighborhood median Tanimoto |
| MolMiner | 0.842 | 0.203 | 0.121 |
| HierVAE | 0.869 | 0.203 | 0.115 |
| GDSS | 0.526 | 0.083 | 0.080 |
MolMiner and HierVAE have chemically cohesive neighborhoods, with AUC near 0.84 and 0.87, well above the 0.5 chance line. GDSS sits near chance (0.526). Its latent coordinate barely determines which molecule is produced, and most of the variation in molecular identity comes from decoder randomness rather than the coordinate itself.
The most important finding is that cohesiveness is decoupled from metric smoothness. HierVAE is strongly cohesive (AUC 0.869), yet Euclidean distance in its latent space barely predicts chemical similarity (R²euc is just 0.05, and cosine is the same); MolMiner partially predicts it (R²euc 0.33). Being locally organized does not imply that interpolating along a coordinate changes the output smoothly.
Boundary structure is coarse-to-fine: early decoding steps carve a few broad regions (corresponding to different initial fragments); later steps subdivide within each region; the coarse boundaries between regions largely persist. During training, chemical cohesiveness stabilizes early, while the number of distinct identities each neighborhood can produce (granularity) keeps changing. A model can be locally organized while still reshuffling what it can actually generate.
The takeaway is useful for anyone doing latent-space optimization or interpolation, not just in molecules: a latent space does not automatically become a smooth, navigable map of the output. You have to verify it yourself. Three concrete points follow. A diffusion model like GDSS may give you almost no coordinate control, so the "latent" is not meaningfully steering the molecule; even when a model is locally organized (high AUC), Euclidean distance can be a poor proxy and gradient-based latent optimization can mislead; checkpoints with similar cohesiveness may generate quite different things. The practical conclusion: before trusting latent-space walks for discovery, test empirically whether the space is navigable at all.