Stanford's Anshul Kundaje calls BioReason's variant effect prediction evals seriously flawed
anshulkundaje · x · 2026-10-23
Stanford professor Anshul Kundaje criticized current variant effect prediction evals as not serious: BioReason's evaluations are "extremely flawed," with unclear handling of train/test leakage, and most other evals are pointless. He argues model builders should instead benchmark against established resources like AlphaMissense, PromoterAI, AlphaGenome, SpliceAI/2 and TraitGym.
More from Research
- VerifierBench: 222 Tasks Expose How Unreliable Agent-Generated Verifiers Are — Jianing Yin · 2026-10-23
- LM2LM: Frozen World Model Provides Scalable RL Rewards for AI Self-Improvement — Minhua Lin · 2026-10-23
- Stanford prof: LLM-fused models need radically new evals to separate memory from reasoning — anshulkundaje · 2026-10-23
- Stanford professor warns fused LLMs blur the line between memorization and real reasoning — anshulkundaje · 2026-10-23
- NSF picks nine teams for up to $3.5M each to boost super intelligence infrastructure efficiency — sh_reya · 2026-10-23
- CMU professor wins NSF-STRIDE AI Efficiency Challenge grant, hiring PhD students and postdocs — sh_reya · 2026-10-23