LongChang benchmark: GPT-6-Astra hits 16.7% discovery rate, other 11 LLMs score zero
rohanpaul_ai · x · 2026-11-01
Researchers unveiled LongChang, an automated benchmark testing whether frontier LLMs can do autonomous scientific discovery: models explore 14 simulated worlds with deliberately altered physical laws, freely manipulating objects, building tools, and choosing their own research questions with no hints—LLM agents handle scene generation, tool review, and report evaluation with no human in the loop.
Key results:
- Only GPT-6-Astra achieved a discovery rate of 16.7%; the other 11 LLMs all scored 0%
- A 5× larger exploration budget brought no improvement
- Exploration records show uninvestigated anomalies and only partial identification of altered laws
Named after the town where philosopher Wang Yangming attained enlightenment. Paper, code, and interactive demo are public.
More from Research
- awesome-ai-constitutions repo collects AI constitutions & model specs with generalization research — dhadfieldmenell · 2026-11-01
- Karpathy's nanochat hits 59K stars: a full LLM training lab in one repo — techNmak · 2026-11-01
- Same 2.4 eV band gap, different routes: SHAP explains per-prediction feature contributions — bravo_abad · 2026-11-01
- Hands-on tutorial trains a decision model on Qwen3-0.6B from scratch — dl_weekly · 2026-11-01
- From-scratch 8M-parameter LLM: dev blogs the whole training journey — the-random-guy-2002 · 2026-11-01
- Fine-grained MoE is the easiest win: DeepSeek-V3-style 128 experts cut loss by up to 6.2% — ivan_bezdomny · 2026-11-01