Stanford's Anshul Kundaje presses SmolLM lead on train-test leakage accounting in benchmarks
anshulkundaje · x · 2026-10-23
Stanford professor Anshul Kundaje publicly questioned SmolLM lead Loubna Ben Allal about how her team accounts for various forms of train-vs-test leakage in benchmark evaluations.
He offered to have a longer conversation, framing the critique as wanting her team to succeed and pointing them in the right direction — a substantive technical dispute over benchmark rigor in open model releases.
More from Models
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Are models in an uncanny valley? Researcher warns users to stay vigilant — danielrock · 2026-10-23
- Uncensored Swift 1.5 Qwen 3.8 Flash Next Released as 4-bit GGUF with Strata-Ready Artifacts — jjusko20 · 2026-10-23
- Rumor: Anthropic training a model matching OpenAI's internal one; big launches in December — imjustnewatai · 2026-10-23
- AI researcher: Claude finally finished tedious research tasks I gave up on years ago — MikePFrank · 2026-10-23
- Dream Labs emerges from stealth with Dream-1 robot foundation model at $3/hour — scott_e_reed · 2026-10-23