Interp debate: superposition may just be concepts you haven't thought of
kuza55 · x · 2026-10-16
A discussion thread on neural network interpretability argues that what looks like superposition may simply be a concept you haven't thought of — the famous cats+cars neuron was actually a "v-shape", making it conceptually meaningful even if not obvious. The thread also notes that agents can be given interp tools to automate this process, filling out most of the network consistently like a jigsaw puzzle. The poster admits limited depth but notes roon considers this line of work important.
More from Research
- UCSD Researchers Propose Dataset Watermarking Detectable via Black-Box API Access — yuxiangw_cs · 2026-10-16
- EmbodiedEvo tops RoboDojo as agents inherit reusable skills and lessons — Aiden_Tech_Ai · 2026-10-16
- CHI reviewer laments: papers' Discussion sections now cite nothing — IanArawjo · 2026-10-16
- Casper: Understanding 90% of neurons ≠ understanding the network — StephenLCasper · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Agentic Search Systems Are Rewarded Only for Final Answers, Raising Faithfulness Concerns — burkov · 2026-10-16