Depth vs hold: behavior alone can't diagnose why LLMs fail, new paper argues
burny_tech · x · 2026-10-13
A paper by Queloz & Beckmann (arXiv:2609.36079) splits model understanding into two dimensions:
- Depth: how accurate and general the model's internal circuitry is.
- Hold: how reliably that circuitry is recruited and stays in control.
Behavior alone can't distinguish shallow understanding from weak hold — they look identical outside but demand different fixes. A 2×2 matrix maps the space: deep+firm grounds trust; deep+precarious is fixed via orchestration; shallow+firm needs deeper pre-training; shallow+precarious is the worst case. Only looking inside the model can tell them apart.
More from Models
- Muse senior engineer benchmark results pulled over flawed run, retest underway — danshipper · 2026-10-13
- "Zero chance" a Meta model competes with Google, OpenAI or Anthropic right now — iruletheworldmo · 2026-10-13
- Dev turns EmbeddingGemma 2 into an on-device STT model, transcribing 1 hour of audio in 13 seconds — solyarisoftware · 2026-10-13
- Alexandr Wang hails Theo's model rankings: Opus 5.5 and Muse Wonder alone in S tier — alexandr_wang · 2026-10-13
- Reddit user rails against Gemini's decline: wrong bulb model, 13 failed script fixes — hotboxuzi · 2026-10-13
- steipete: 6.1 Sol ultrafast beats 6 Astra — same cost, but 250 tpm changes everything — steipete · 2026-10-13