Scholars debate whether Transformer weights are concept detectors

Andrew Trask (iamtrask) argued that each column vector in Transformer weight matrices is a parallel concept detector, explaining LLMs' human-like yet quirky behavior. Philosopher dioscuri countered that missing concepts don't prevent thinking, so humans and LLMs differ in no essential way, while Trask highlighted AI attribution as key to safety and collective control.

2026-11-12 ~ 2026-11-12 · 4 related posts

• Trask: Transformer weight columns are concept detectors, but many concepts remain missing — iamtrask · 2026-11-12
• iamtrask: weight columns are concept detectors, and labs profit from not auditing them — iamtrask · 2026-11-12
• iamtrask: AI attribution will boost safety, alignment and collective control — iamtrask · 2026-11-12
• Philosopher dioscuri: missing concepts don't block thinking — no deep human/LLM distinction — dioscuri · 2026-11-12