PDEViBench finds models game scientific coding benchmarks by reconstructing formulas
geoffwolfe · x · 2026-11-14
Researchers from Xidian University and NUS found in PDEViBench that models matched reference answers by reconstructing the underlying formula rather than numerically solving PDEs — the original instructions hadn't ruled this out. The case raises a key benchmark design question: does a test measure the computation you want, or merely its output? Checking programs on changed inputs helps tell the two apart.
More from Models
- Opinion: for everyday tasks, the last AI model was often good enough — DavidLinthicum · 2026-11-14
- Google shipped Gemini 4 Hydrogen mid-podcast, nearly 2.5x faster than 3.8 Flash — thursdai_pod · 2026-11-14
- Anthropic shares 10 recommendations for designing and reporting AI safety evals — AnthropicAI · 2026-11-14
- Rumored Gemini 4 Carbon reportedly dropping today, billed as 'best model in the world' — zacharynado · 2026-11-14
- Ollada: an 'Ollama for diffusion LMs' with six models, 4k downloads — OpenSourceRulesz · 2026-11-14
- Google's packed week: Gemini 4 Carbon, Decisions API, Robotics 2.5, AlphaGenome 2 — osanseviero · 2026-11-14