Protocol Blocks Over 72% of p-Hacking Transfer Across Two Tasks
krisgligoric · x · 2026-07-06
The protocol was evaluated on two tasks with "zero true effects," meaning any significant result was considered p-hacking. The testing covered 20 models, 4 providers, and 11 configurations. The protocol successfully blocked 73.9% and 72.7% of p-hacking transfers across the two tasks, respectively.
Related event: Study Proposes Preregistration Protocol to Curb LLM Evaluation P-Hacking(7 posts)→
More from Research
- OpenAI internal model claims to refute Exponential Time Hypothesis; independent review organized — Singularitarian · 2026-10-27
- Music Lenses: ISMIR paper uses sparse autoencoders for steerable concept-level music retrieval — umpedronosapato · 2026-10-27
- Neuronpedia adds 7M+ neurons for Qwen3-32B, Gemma and Llama 3.3 70B to its interpretability encyclopedia — aryaman2020 · 2026-10-27
- Matthew Green hails stunning new ETH result pending Lean proof validity — matthew_d_green · 2026-10-27
- Heuristic for prediction accuracy: a bimodal probability histogram peaking at 0 and 1 — moultano · 2026-10-27
- AI agent autonomously discovers and implements high-order Maxwell solver scheme — burny_tech · 2026-10-27