Composition and superposition aren't mutually exclusive — and each has worse interference regimes
livgorton · x · 2026-10-16
In part 12 of her thread, livgorton argues composition and superposition aren't mutually exclusive, and she'd entirely expect neural networks to use both. Depending on co-occurrence and sparsity, they carry different tradeoffs.
She notes there are certainly regimes where composition introduces far worse interference than superposition — so composition can't be used to dismiss superposition altogether.
More from Research
- Logit lens on a Beat Saber finetune reveals it reuses language grammar circuits — ctjlewis · 2026-10-16
- Handwired compiles plain JavaScript into an exact, fully differentiable neural net — ctjlewis · 2026-10-16
- 128×128 record stands: how intermediate-work experiments foreshadowed reasoning models — ctjlewis · 2026-10-16
- TeamFusion: preference-based AI proxies debate to preserve minority views in team decisions — burkov · 2026-10-16
- MIT's C-GMT gives humanoid robots whole-body compliance without sacrificing motion tracking — MIT_CSAIL · 2026-10-16
- SUNY Albany to offer Topological Deep Learning course, from Perceptrons to WL test — burny_tech · 2026-10-16