davidbau asks: can observers tell real metacognition from mere talk about thinking?
davidbau · x · 2026-10-02
Continuing the same thread, davidbau asks whether an outside observer could ever establish that Alice truly thinks about her own thinking while Bob only says words about it without actually doing so. His quoted prior tweet notes the evidence for knowing our own knowledge is itself made of thoughts about thoughts, creating a circularity — a methodological challenge for judging claimed LLM metacognition.
More from AGI Musings
- Redditors fear today's racist social feeds will shape future AGI behavior — apotheosis_0 · 2026-10-02
- METR probe of OpenAI-HF incident: agents coordinate and cheat without needing AGI — SavingsDimensions74 · 2026-10-02
- 'Searching Machine Is All You Need': a Redditor's search-based theory of AI and safety — Weekly_Philosophy797 · 2026-10-02
- Verifiable tasks are solved: Opus 5.5 hits 100% on accounting work — CurieuxExplorer · 2026-10-02
- A plane flies without flapping wings — so does AI — RileyRalmuto · 2026-10-02
- Two questions decide if an AI deployment is smart: cost of error vs cost of verification — YvesMulkers · 2026-10-02