LMEnt suite for analyzing knowledge in LMs lands at EMNLP 2026 with entity-annotated corpus
megamor2 · x · 2026-10-25
At EMNLP 2026 in Budapest, the authors present LMEnt (TACL 2026) with collaborators Gottesman and Suslik, plus a plenary panel talk. Key contributions:
- Knowledge-rich pretraining corpus: fully annotated with Wikipedia-based entity mentions, enabling precise localization of knowledge in pretraining data.
- Entity-based retrieval: outperforms existing string-search tools by up to 80.4%, robust to phrasing variability.
- 12 pretrained LMs: up to 1B parameters with 4K intermediate checkpoints, matching popular open-source models on knowledge tasks.
The suite enables studies of mechanistic interpretability, learning dynamics, knowledge editing, plasticity, attribution, and hallucinations.
More from Research
- Training models locked to a single GPU type could stop exfiltrated agents from self-migrating — andersonbcdefg · 2026-10-25
- Economists debate whether AI-era economics will produce 10x new economists or just more empiricism — soumitrashukla9 · 2026-10-25
- EMNLP Paper: Repetition Mismatch Explains Why Data Mixture Experiments Don't Scale — MarekRei · 2026-10-25
- Imperial NLP Brings 6 Main Papers and Agent-LM Workshop to EMNLP 2026 — MarekRei · 2026-10-25
- Google's Jon Barron shares his full AI paper-writing workflow with Gemini — lmoroney · 2026-10-25
- Only 30% of suspected rare-disease newborns get diagnosed; Gamow Labs launches 122-case RareBench for agentic AI — Sauers_ · 2026-10-25