OfirPress marks 10 years in language modeling; his weight tying paper was once rejected as unrealistic
OfirPress · x · 2026-09-06
OfirPress reflects on a decade in language modeling. His first paper (started Jan 2016) showed the embedding and unembedding layers could share one matrix. It was rejected by EMNLP 2016 with a reviewer calling language modeling unrealistic, later accepted at EACL 2017, and weight tying is still used in smaller models today.
Related event: Ofir Press Marks a Decade of Language Modeling with Old Rejection Letter(2 posts)→
More from Companies & People
- Tesla Robotaxi becomes the #1 travel app, surpassing Uber as Uber allies with taxi drivers — SumitGup · 2026-09-06
- Anthropic forced to pay ~5x to buy out compute contracts as OpenAI locked up supply — haider1 · 2026-09-06
- Sam Altman: Curing Cancer With AI Is 'Not Enough', Industry Should Aim Higher — SydSteyerhart · 2026-09-06
- Larry Ellison: A lot of Oracle's code is now written by its AI models, not people — r0ck3t23 · 2026-09-06
- Karpathy joins Anthropic: 700 experiments in 2 days, knowledge graphs boost his loop — blaizedsouza · 2026-09-06
- AI math recruiting: papers, awards and agent experience weigh heaviest — _sathvikr · 2026-09-06