LLMs Exacerbate p-hacking in Scientific Research
krisgligoric · x · 2026-07-06
The author notes that while p-hacking is a long-standing issue in the scientific community, LLMs (when used as annotation tools or LLM-as-a-judge) make it even easier to achieve desired results simply by tweaking parameters. The author proposes a mitigation method and verifies its effectiveness through a pre-registered study.
Related event: Study Proposes Preregistration Protocol to Curb LLM Evaluation P-Hacking(7 posts)→
More from Research
- Nature Physics: reprogrammable refractive index enables arbitrary multimode wave control for ML — MarioKrenn6240 · 2026-10-27
- Wakaru open-sources ontologize: labeling datasets from model activations — amplifiedamp · 2026-10-27
- EMNLP Oral paper traces multilingual representations in LLMs with cross-layer transcoders — ZhijingJin · 2026-10-27
- GPT-6 Pro reruns crime research: GTA V's 2013 launch tied to violence dips in four regions — emollick · 2026-10-27
- OpenAI's Aaron Lou on leaving a Stanford PhD for OpenAI, diffusion LMs vs autoregression — aaron_lou · 2026-10-27
- CLT-Forge: Open-source toolkit for cross-layer transcoder training and attribution graphs — ZhijingJin · 2026-10-27