LLMs learn atom types but not geometry: benchmark scores can mask missing structure
bravo_abad · x · 2026-10-13
Alampara et al. built synthetic datasets where answers depended either on particle types or inter-particle distances, finding LLMs learn composition far more readily than spatial relationships. In crystal-property benchmarks, text with explicit atomic coordinates performed no better than simple local-environment descriptions, and scaling fine-tuned Llama-2 from 7B to 70B still left a gap versus graph neural networks that use real geometry. Benchmark scores don't prove a materials model distinguishes atomic arrangement — the key differentiator between diamond and graphite.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23