Autoregressive vs discriminative models: distribution mismatch drives generalization failure
ysu_nlp · x · 2026-09-24
A comparison study shows autoregressive models (ArcaneQA) skew heavily toward seen data, creating distributional mismatch and generalization failure from seen to unseen cases, while discriminative models (Pangu) produce much better-aligned output probability distributions over seen and unseen data. Paper linked in the post.
More from Research
- Anthropic's wet lab makes first discovery: ~950 Claude agents find new gene-editing mechanism in 21 hours — rohanpaul_ai · 2026-09-24
- Zeta(5) claimed irrational: Lean 4 formalization of the proof published on GitHub — AlexKontorovich · 2026-09-24
- Gaia paper: metagenomic discovery with late-2024 LLMs predates Claude's find — owl_posting · 2026-09-24
- Yoav Goldberg: LLM Spotted the Pattern by Analogizing It to CRISPR — yoavgo · 2026-09-24
- TRACES: A New Benchmark That Grades AI Problem-Solving Process, Not Just Correct Answers — dr_cintas · 2026-09-24
- ECCV MMBU Benchmark Shows VLMs Answer Biomedical Questions Without Knowing What They See — davidjhwu · 2026-09-24