Understanding as a Competency Vector: Prediction, Explanation and Control Trade Off
burny_tech · x · 2026-08-28
The author defines "absolute understanding" of a system as the theoretical maximum of predictive, explanatory, interventional and control competence — a limiting ideal rarely achievable given chaos, computational irreducibility and undecidability, so practical understanding is always a spectrum.
Key points:
- Black-box statistical models can predict extremely well while modeling little causal structure — a genuine trade-off with mechanistic understanding; the predictor's internal mechanism may partly mirror the external system's
- Control is formally the ability to find an action sequence moving the system into a target region, which requires correctly predicting counterfactual interventions
- Models differ in scope, generalization, adaptivity, efficiency and robustness
- Understanding is thus a multidimensional competency vector, not a scalar; given trade-offs, multiple maximally-understanding models may exist, forming a Pareto frontier
More from AGI Musings
- Why Did GPT-5.5 Love Goblins? Unpredictable AI Alignment Sparks Debate — louisvarge · 2026-08-28
- Opinion: Pause Pretraining to Focus on Controlling Inner Model Drives — louisvarge · 2026-08-28
- Thought experiment: Pause pretraining to focus on controlling inner drives — louisvarge · 2026-08-28
- Judea Pearl: gene-IQ debate can be settled the same way as smoking-cancer was — yudapearl · 2026-08-28
- AI Reliability Lags 4-10x Behind Capabilities; 90% Accuracy Is Catastrophic — sayashk · 2026-08-28
- Google DeepMind uses Teamwork multi-agent framework for math breakthroughs — algo_diver · 2026-08-28