Liquid AI lifts the decoder's causal mask: bidirectional attention beats causal for decision models
helloiamleonie · x · 2026-11-06
Liquid AI's Leonie explains their latest release's core design choice: use a decoder-based approach but lift the causal mask. The mask makes sense for next-token prediction, but why shouldn't a decision model see all tokens? With bidirectional attention over full context they get a model trained to make decisions — though overconfidence follows.
More from Models
- Bittensor miners train open-source decision model REAX 1.3 — const_reborn · 2026-11-06
- Bittensor subnet 92 releases REAX 1.3, an open-source decision model blending miner models — const_reborn · 2026-11-06
- Kimi K3.1 Pro scores 9.9% on agentic WeirdML v3, ahead of Kimi-K3 but behind Claude Haiku 5.5 — scaling01 · 2026-11-06
- Rumor: GLM-5.5 to come in 32B and 122B sizes, community figure says no — TheZachMueller · 2026-11-06
- User screenshot suggests Anthropic's weekly Claude limits feel effectively unlimited — StewartalsopIII · 2026-11-06
- Leonie Shares Full Slide Deck on Building Decision Models, Still Early Days — helloiamleonie · 2026-11-06