Counterfactual propagation as a measure of "structural" conjectures, and why AI finds counterexamples but struggles with theory-building

GlenBradley · x · 2026-08-25

A thoughtful reply to Eric Weinstein's thread on the S^6 integrable complex structure result.

The author proposes characterizing whether a conjecture is "structural" via counterfactual propagation: temporarily impose ¬C on the epistemic dependency graph around C, and measure how much neighboring mathematics must be revised — call the affected subgraph Δ(C). An isolated conjecture has small Δ(C); a genuinely structural one has a large, deep, highly connected Δ(C), where falsifying it creates dissonance across substantial adjacent mathematics.

This suggests an interpretation of the current AI asymmetry: finding a counterexample requires locating a witness w with w ⟹ ¬C, often with a crisp verification condition. Theory-building requires following the resulting dissonance through Δ(C) and reconstructing a coherent graph — much harder. Perhaps what we are watching is machines becoming very good at the former.

Original post →

Counterfactual propagation as a measure of "structural" conjectures, and why AI finds counterexamples but struggles with theory-building

More from AGI Musings