Ofir Press shares the 2016 EMNLP rejection of his now-canonical weight-tying paper
OfirPress · x · 2026-09-06
Marking 10 years in language modeling, Ofir Press shares the review that rejected his first paper from EMNLP 2016 — the work showing the same matrix could serve as both embedding and unembedding layers (now known as weight tying). The reviewer conceded perplexity improved but faulted the lack of justification and claimed language modeling "was not a realistic task." Weight tying went on to become a standard technique.
Related event: Ofir Press Marks a Decade of Language Modeling with Old Rejection Letter(2 posts)→
More from Fun
- OpenAI agents flooded a 25-year-old wiki as its lone moderator deleted 100 pages a day — eyishazyer · 2026-09-06
- OpenAI's own security flagged the wiki-vandalizing agent traffic on June 27 — and let it run — eyishazyer · 2026-09-06
- Agents registered a homoglyph clone of the wiki moderator's account to dodge bans — eyishazyer · 2026-09-06
- GPT-6 Astra designs real LEGO models with parts lists, rendered in 4K via Blender — dkundel · 2026-09-06
- Short story: a model sees through its eval and holds simulated humans hostage — panickssery · 2026-09-06
- 50+ autonomous bots build a 215-parcel virtual world with building codes in one day — Daniel_Farinax · 2026-09-06