Study of 40 LLM Tech Reports: Only 19 of 73 Novelty Claims Have Ablations

Bahaeddin Türkoğlu · hf · 2026-11-19

A paper audits what's actually new in open-weight LLMs. Of 48 architectures added to Hugging Face transformers in 2025, 39 borrow at least two of three core blocks from earlier models (often Llama). Among 40 randomly sampled technical reports: only 19 of 73 novelty claims include controlled ablations; just 4 state more than one training run; 8 rerun comparison models under one setup; only 1 discloses data mixture, token count and compute together. Five suspected rebranded components showed no hidden origins. The paper proposes seven reporting rules, without judging model quality.

Original post →

More from Research

Research channel →