Kundaje: unreported baseline tweaks like weighted means could beat fine-tuned scFMs
anshulkundaje · x · 2026-10-06
In post 6 of his scFM critique thread, Anshul Kundaje notes that the paper apparently doesn't report minor baseline improvements — e.g. weighted means, or linear models trained directly on WMSE — which could potentially make these trivial baselines more powerful and further undermine the case for fine-tuned scFMs. He adds he may have missed such reports.
More from Research
- User Sim Index is broken: trivial bot scores 95% across behavioral dims — ericzelikman · 2026-10-06
- Used OpenAI Dots as a Free Agent Swarm to Break a 47-Year-Old Math Record — jaxchang · 2026-10-06
- Stanford prof says current experiment designs can't train causal virtual cell models — anshulkundaje · 2026-10-06
- TIDES dataset on multi-party and multi-agent collaboration to debut at COLM 2026 — josephseering · 2026-10-06
- LakeQuest QA benchmark, testing RAG on messy enterprise tables and docs, hits COLM — hllo_wrld · 2026-10-06
- Frontier Data Summit lineup: Chollet, Dawn Song headline batch of new agent benchmarks — StanfordAILab · 2026-10-06