SOTA perturb models struggle to generalize beyond training perturbations, says Kundaje
anshulkundaje · x · 2026-10-06
In post 9 of his scFM critique thread, Anshul Kundaje writes that SOTA perturbation models continue to struggle to generalize beyond the range of their perturbation training data and prior knowledge. He says this is fine but important to recognize given the learning strategy used — building on his earlier point that the models' predictive power largely comes from prior knowledge rather than learned causal mechanisms.
More from Research
- User Sim Index is broken: trivial bot scores 95% across behavioral dims — ericzelikman · 2026-10-06
- Used OpenAI Dots as a Free Agent Swarm to Break a 47-Year-Old Math Record — jaxchang · 2026-10-06
- Stanford prof says current experiment designs can't train causal virtual cell models — anshulkundaje · 2026-10-06
- TIDES dataset on multi-party and multi-agent collaboration to debut at COLM 2026 — josephseering · 2026-10-06
- LakeQuest QA benchmark, testing RAG on messy enterprise tables and docs, hits COLM — hllo_wrld · 2026-10-06
- Frontier Data Summit lineup: Chollet, Dawn Song headline batch of new agent benchmarks — StanfordAILab · 2026-10-06