Using Websites to Aggregate Research Problems as Benchmarks
kylekabasares · x · 2026-07-17
- The retweeted content argues: If you want a particular field of research to truly advance, centralize the unsolved problems on a dedicated website to direct attention and compute power toward them.
- Examples include the Erdős problems in mathematics and open problem lists in statistics/machine learning.
- Core viewpoint: While there are already many benchmarks for AI's progress on math problems, what's truly more valuable is whether it can help researchers solve real research questions that "have no known answers yet."
- Therefore, the author encourages all fields to build similar "problem aggregation sites" to systematically collect unsolved problems.
More from AGI Musings
- Claude Code skill uses 10 Markdown rules to make outputs ADHD-friendly — alex_verem · 2026-07-22
- ControlAI CEO says an international ban on superintelligence is needed to avert extinction risk — zetalyrae · 2026-07-22
- Gary Marcus says LLMs still cannot really do math on their own — GaryMarcus · 2026-07-22
- Gary Marcus says LLM math skills are like knowing only a car’s engine size — GaryMarcus · 2026-07-22
- AI may make digital work infinitely leveraged while offline life gets more human — illscience · 2026-07-22
- Better AI math could save researchers time by killing false conjectures earlier — prateekj · 2026-07-22