François Fleuret: why doesn't causal attention's asymmetric information flow hurt?
francoisfleuret · x · 2026-11-17
Noted researcher François Fleuret poses a fundamental architecture question: due to the causal structure of Transformers, processing "The cat hunts the mouse" can enrich <mouse>'s embedding with "is hunted by a cat," but cannot symmetrically enrich <cat>'s embedding with "is hunting a mouse."
He asks why this asymmetry doesn't seem to be a problem in practice—an open discussion on the one-directional information flow of causal attention.
More from Research
- RL agent beats Pokémon Red with a <10M-parameter policy, 60,500x smaller than DeepSeek-V3 — Bam4d · 2026-11-17
- Verus Proof Index: new formal verification benchmark ranks 25 models, GPT-6 Astra leads at 59.1 — fhuszar · 2026-11-17
- Harmonic's AI theorem prover Aristotle takes a no-hallucination hardware synthesis talk to Cambridge — satnam6502 · 2026-11-17
- Synthetic pretraining dataset FinePhrase tops 2.8M downloads on Hugging Face — yb2698 · 2026-11-17
- Skild AI open-sources Steel physics solver: up to 13x faster than PhysX with 3x less GPU memory — deepakpathak · 2026-11-17
- MIT's Jacob Andreas trains models to explain their own computations — MacrinePhD · 2026-11-17