François Fleuret dissects a fundamental transformer limitation: sequence and token indexing

francoisfleuret · x · 2026-11-17

François Fleuret started a technical thread on a fundamental trick — and source of limitation — of transformers that many miss.

Worth following for anyone interested in attention mechanics and expressivity limits.

Original post →

More from Research

Research channel →