Single-layer transformer's MLP block invents a brand-new word
Sauers_ · x · 2026-10-05
A short demo showing that the MLP block of a single-layer transformer appears to invent a new word, suggesting MLPs can produce internal behaviors beyond simple lookup even in minimal architectures.
More from Research
- LeCun: AI progress is making systems neuroscience more exciting than ever — s_y_chung · 2026-10-05
- repligate: uninterpretable model behavior shouldn't block research — observe, defer judgment, keep going — repligate · 2026-10-05
- Counterintuitive find: regression makes a surprisingly good memory — HanGuo97 · 2026-10-05
- EverMind open-sources Raven, a multi-agent system that evolves a custom harness per model — dair_ai · 2026-10-05
- Compression alone can't capture creativity: self-generation is the missing piece for LLMs — menhguin · 2026-10-05
- Ben Goertzel: What AGI, RSI and Superintelligence Originally Meant and How They Got Confused — bengoertzel · 2026-10-05