Dietterich asks: do LLMs learn universal rules like x=½at², or just interpolate?
tdietterich · x · 2026-10-11
ML pioneer Thomas Dietterich responds to Andrew Wilson's question about unexplained aspects of generalization with a concrete standard: given (x,t) pairs of an object under constant acceleration, can a model recover x = ½at² as a universally generalizing rule?
His two core questions:
- Do LLMs achieve this rule-discovering kind of generalization?
- Or are they merely doing kernel interpolation on the data pairs?
He notes in a follow-up that symbolic regression methods already solve this, but he is specifically interested in neural network generalization — a pointed academic challenge about whether deep learning truly generalizes or just interpolates.
More from Research
- Paper: LLMs are too agreeable in medical diagnosis, sycophancy hurts accuracy — lulzxdxdxd · 2026-10-11
- Hitchhiker's Guide to the AI Data Galaxy: 10 Key Terms from RL Environments to pass@1 — geoffwolfe · 2026-10-11
- Sherlock game shows small-agent swarm plus strong reasoner solves 96/100 murder cases — No_Yogurtcloset_7050 · 2026-10-11
- Open-source Blue-Eye classifier claims to beat Google and AWS at NSFW detection for a fraction of the cost — Mundane_Toe_8074 · 2026-10-11
- Internalizer hypernetwork turns documents into LoRA adapters for a 284B model — teortaxesTex · 2026-10-11
- Dev trains a 9B Qwen3.5 base model on SVG generation to test a new training framework — cephaloform · 2026-10-11