Stop Stacking Complex Modules: Vanilla Transformer Nails Small Molecule Docking
alex_peys · x · 2026-08-13
A researcher criticized the academic incentive in ML (especially Bio+ML) to add unnecessary moving parts to models just to make them look fancy. These tricks often add no performance value, make scaling harder, and send others on wild goose chases.
Curious about the limits of simple architectures, they tested this on small molecule docking: using a vanilla transformer that takes a SMILES string and rigid pocket backbone to directly diffuse 3D coordinates of the ligand.
Key Approaches & Results:
- Skipped complex designs like bond losses, equivariance, pair representations, and local frames, relying only on random rotations.
- Used a strategy of pre-training on drug-like molecules without pockets (massive data is key), then fine-tuning on real interactions.
- The simple model works great and scales exceptionally well.
More from Research
- New BPJ jailbreak bypasses top defenses with single-bit black-box attacks — StephenLCasper · 2026-08-13
- Paper proposes safety case framework for AI misuse safeguards — StephenLCasper · 2026-08-13
- Google deploys production-ready probes for Gemini, tackling long-context shifts — StephenLCasper · 2026-08-13
- ICLR paper: Filtering pretraining data makes open-weight LLMs tamper-resistant — StephenLCasper · 2026-08-13
- Paper examines when capability restrictions are warranted to prevent AI misuse — StephenLCasper · 2026-08-13
- MLCommons Releases Enterprise Guide to Spotting AI Benchmark Washing — TheKanter · 2026-08-13