Trask: Transformer weight columns are concept detectors, but many concepts remain missing
iamtrask · x · 2026-11-12
Andrew Trask (iamtrask) argues that each column vector in a transformer's weights acts as a parallel concept detector, with lower-level concepts detected early in the network and combined by higher-level detectors later.
These vectors start semantically empty and get filled by patterns from web scrapes, purchased data, user logs and human labeling. But pockets remain missing because humans don't write down all the concepts they use and their interconnections — or frontier labs haven't acquired that data yet.
Related event: iamtrask: Transformer Weights Are Concept Detectors(3 posts)→
More from Research
- Meta Releases XRStereo: ~18k-Sequence Synthetic Stereo Dataset on Hugging Face — _akhaliq · 2026-11-12
- iamtrask: an LLM is learning to fill in a giant next-token frequency matrix — iamtrask · 2026-11-12
- Qwen 3.8 27B on a laptop can read model weights, replicating the Weight-Space Programming paper — dl_weekly · 2026-11-12
- Dense rewards lose to standard GRPO at matched compute, arXiv paper finds — burny_tech · 2026-11-12
- Wayve unveils Learned Safety Envelope: end-to-end trained live risk prediction for AI driving — alexgkendall · 2026-11-12
- HipKittens turns one: breaking AMD's 'CUDA moat' for a multi-silicon future — simran_s_arora · 2026-11-12