Tom Dietterich: 'Albany is in NY' is simple — its web of associations is not
tdietterich · x · 2026-09-28
In a discussion on how LLMs represent knowledge, veteran ML researcher Tom Dietterich argues that a fact like "Albany is in New York State" is fairly simple on its own. What is not simple, he says, is the vast array of associations connected to Albany and the situations in which those associations become salient for a given task. He notes such facts can be stored in a database, but representing all the associations is where neural networks shine.
More from AGI Musings
- Tegmark backs 2026 Singapore Consensus, a roadmap of global AI safety research priorities — tegmark · 2026-09-28
- Data labeling firm sees Fortune 1000 revenue shift and evals as core enterprise IP — Scobleizer · 2026-09-28
- NYU professor: LLMs fundamentally solved common sense; physical world is next — ipeirotis · 2026-09-28
- OpenAI's Logan Kilpatrick: revenue-generating agent companies arrive at scale in 2027 — OfficialLoganK · 2026-09-28
- Ackman vs quant: AI compute demand may break the Fed's inflation playbook — Afinetheorem · 2026-09-28
- AI practitioner warns: we may eat away agents' productivity gains by just getting lazier — trq212 · 2026-09-28