SemiAnalysis launches GDPx benchmark: best model scores just 12%, and GPT-6 Pro found grading errors
eldonredwards · x · 2026-11-03
SemiAnalysis released GDPx, a frontier-level benchmark measuring agentic capabilities across the most valuable domains — the best model scores only 12% (1/7).
Ethan Mollick welcomed it as a successor to GDPval but noted it needs human baselines; he says GPT-6 Pro spotted errors in the grading, and that 0% scores hide economically meaningful differences in success rates.
Attached screenshots analyze two concrete grading issues: a verifier reporting four customer-program adjustments as missing when they exist in a supporting tab — likely a presentation-compliance failure, not a comprehension failure; and a simulator bug where the 350-microsecond IEX delay is applied to order placement but omitted for cancellations, contradicting IEX docs, meaning the winning trading strategy's pass may be a simulator artifact.
Related event: SemiAnalysis Launches GDPx Benchmark; Best Model Scores Just 12%(17 posts)→
More from Models
- Zyphra's ZHA brings subquadratic attention via hierarchical tree-structured indexer — yacineMTB · 2026-11-03
- LiquidAI's LFM3-1.7B tops HF trending with 1k likes and 42.4k downloads in 5 days — maximelabonne · 2026-11-03
- 'Gemini is your friend and alignment is complicated': another eyebrow-raising model reply — vedantmisra · 2026-11-03
- Pedro Domingos: Humans are much more resistant to data than AI is — pmddomingos · 2026-11-03
- OpenAI says search-like queries in ChatGPT are one-fifth of Google's volume — firstadopter · 2026-11-03
- LFM3-1.7B quantization report: near-lossless at 3.6 GB VRAM, 2.2 GB with modest degradation — crusaderky · 2026-11-03