Ex-OpenAI Insider: Every Possible Attention Trick Was Tried in 2021
dejavucoder · x · 2026-08-03
Noted AI community figure @tszzl mentioned in a discussion that every conceivable attention trick was already experimented with at OpenAI back in 2021, sparking reflections on early LLM architecture exploration.
More from Companies & People
- AI Safety Org Resolution Raises $160M, Seeks COO — Astral Codex Ten · 2026-08-04
- NVIDIA and KAIST Launch Joint AI Lab to Advance Agentic AI in Korea — hyunw_kim · 2026-08-04
- From Cost Center to Revenue Engine: The Applied AI GTM Path for PMs — brandon_galang · 2026-08-04
- Wall Street's AI Move: Exposing the Workflow Rather Than Replacing Workers — dfinke · 2026-08-04
- IBM and Sarvam Partner to Advance Sovereign AI for Indian Government — Paimaamu · 2026-08-03
- Spark Capital VC Arpan Sengupta Joins OpenAI to Lead Enterprise Efforts — sherwinwu · 2026-08-03