Scholar Slams AI Evaluation Double Standards, Says 1% Hit Rate Can Drive Science

RexDouglass posted a series of updates directly calling out the severe "double standards" in current AI evaluation metrics, while re-examining the reliability of academic research and the practical value of AI. He argues that the outside world often sets standards for agentic workflow so strictly that even humans cannot meet them. He believes this debate is essentially a culture war disguised as rigor, and that AI's real impact is on jobs rather than anything else.

已确认

观点归属: The above insights come from @RexDouglass's series of posts, which systematically criticize the current state of AI evaluation and academia.
学术基线: He emphasized that in reality, the vast majority of papers across many fields might be wrong. Given the difficulty, having only 1 out of 100 papers be correct is an acceptable baseline.
价值标准: He advocates that the standard for judging AI's value should not be absolute perfection. As long as there is a strictly reproducible core in the research, even a model with a 1/100 or 1/200 hit rate remains highly practical.

尚未确认

The claims that "the vast majority of papers are wrong" and "only 1 in 100 papers is correct" are critical premises and subjective judgments based on the author's observations, not empirically verified statistical data.

为什么重要

This perspective breaks the traditional mindset that "AI must be highly accurate to be useful." By acknowledging the high failure rate of scientific exploration and shifting the focus to "hit rate" and "reproducibility," it provides a highly practical lens for evaluating AI's real-world utility in research and workflows.

2026-07-29 ~ 2026-07-29 · 5 related posts

Primary sources

AI comparisons are rigged, and jobs may be the real casualty — RexDouglass · 2026-07-29 00:14 (UTC+08:00)
AI may crack the replicable core of research with just 1-in-100 accuracy — RexDouglass · 2026-07-29 00:25 (UTC+08:00)
Scholar Critiques Academia: Most Papers Are Flawed, Yet Presented as Absolute Truth — RexDouglass · 2026-07-29 00:05 (UTC+08:00)

[source] Scholar Critiques Academia: Most Papers Are Flawed, Yet Presented as Absolute Truth — RexDouglass · 2026-07-29
A research thread argues most papers are wrong and should not be read literally — RexDouglass · 2026-07-29
[source] AI comparisons are rigged, and jobs may be the real casualty — RexDouglass · 2026-07-29
AI may crack the reproducible core of research with only a 1-in-100 hit rate — RexDouglass · 2026-07-29
[source] AI may crack the replicable core of research with just 1-in-100 accuracy — RexDouglass · 2026-07-29