Sheaf-theoretic decomposition of frozen LLMs beats LEACE fact retrieval by +17.1pp
Bryce Grant · hf · 2026-10-28
A new paper from Case Western Reserve University (Bryce Grant, Peng Wang) applies cellular sheaf theory to decompose frozen language model representations.
- Each paraphrase pair is treated as a local witness that two activations should agree; a cellular sheaf is built over the paraphrase graph, and its Laplacian answers whether representations keep the same content across phrasings.
- The kernel H⁰ is a content subspace: at just 20 dimensions it beats LEACE's preserved subspace on held-out CounterFact fact retrieval across eight architectures by a mean +17.1pp, with joint paraphrase constraint adding +6.5pp.
- The spectral complement concentrates causal influence, revealing architectural fragility via harmonic analysis.
A geometry-plus-spectra toolkit for interpretability.
More from Research
- Dumb chunk-and-rerank baseline beats top agentic memory systems in 6 of 7 benchmarks — JeremyCMorgan · 2026-10-28
- Microsoft's MANDATE: 23 failure categories from 163k Reddit posts on AI agents — dair_ai · 2026-10-28
- Classic cheat sheet: functions, style and rules of thumb for every research paper section — Faheem_uh · 2026-10-28
- Duplicate: OpenAI Scoops Humans on Quantum Math Problem — scientificamerican · 2026-10-28
- BlackboxNLP 2026 program announced: three keynotes, six orals at EMNLP — hanjie_chen · 2026-10-28
- Postdocs Release AI-Assisted Partial Yang-Mills Progress on GitHub After OpenAI Claim — doublefelix921 · 2026-10-28