Self-Test Shows Preregistered Trials Blocked 85.7% of Past p-Hacking
krisgligoric · x · 2026-07-06
The research team also executed and preregistered their own experiments using the protocol. Results showed that 85.7% of configurations previously p-hacked on older models were successfully blocked when using the next eligible model, validating the method's effectiveness for self-regulation.
Related event: Study Proposes Preregistration Protocol to Curb LLM Evaluation P-Hacking(7 posts)→
More from Research
- OpenAI internal model claims to refute Exponential Time Hypothesis; independent review organized — Singularitarian · 2026-10-27
- Music Lenses: ISMIR paper uses sparse autoencoders for steerable concept-level music retrieval — umpedronosapato · 2026-10-27
- Neuronpedia adds 7M+ neurons for Qwen3-32B, Gemma and Llama 3.3 70B to its interpretability encyclopedia — aryaman2020 · 2026-10-27
- Matthew Green hails stunning new ETH result pending Lean proof validity — matthew_d_green · 2026-10-27
- Heuristic for prediction accuracy: a bimodal probability histogram peaking at 0 and 1 — moultano · 2026-10-27
- AI agent autonomously discovers and implements high-order Maxwell solver scheme — burny_tech · 2026-10-27