Debiasing length preference drops a decision model from 60 to 45 on Habermas Machine
antoine_chaffin · x · 2026-10-29
The author debiased their decision model's preference for longer options, dropping it from 60 to 45 on the Habermas Machine benchmark — below the "pick the longest" heuristic (50%, vs 24% for random chance).
Key point: the model wasn't made worse; debiasing prevents it from collapsing when a few tokens are added to the wrong options. The author accepts people may ignore the model over lower benchmark scores, highlighting the tradeoff between leaderboard scores and robustness.
More from Research
- Albert Gu's new H-Net hits 3-5x efficiency gains learning directly from raw bytes — stochasticchasm · 2026-10-29
- DNA motif search tool adds sequence search linked to tripartite graph and ESM2 embeddings — jmschreiber91 · 2026-10-29
- Interactive filters for TF–motif interaction data: family, source, and confidence — jmschreiber91 · 2026-10-29
- Preprint: Human peer reviewers caught only 27% of major errors, all 15 AI models beat them (53-96%) — anshulkundaje · 2026-10-29
- New paper rules out Hamiltonian Phase States as a route to quantum-only Microcrypt — StefanoGogioso · 2026-10-29
- MSR AI for Science team hiring postdoc to generate molecular data for Skala — FrankNoeBerlin · 2026-10-29