Astra audits economics replication code, finds errors overturning journal results
soumitrashukla9 · x · 2026-09-06
- Quoting @cremieuxrecueil's experiment: Google's Astra was run on a batch of academic replication packages and uncovered a huge number of coding errors.
- Most were inconsequential, but some overturn central results of papers in high-ranking journals; Astra also found many models weren't even run properly.
- Greg Brockman amplified it, calling Astra a good tool for checking scientific papers.
More from Models
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- TypeSafe's Jev: a "Decision Model" LLM That Returns Floats at $0.042/M Input Tokens — ariG23498 · 2026-10-23
- Microsoft's MAI-Base-2 scaling ladder pegs the L42 run at ~1.8×10^23 FLOPs — teortaxesTex · 2026-10-23
- Local model agentic coding benchmark: 27B matches 125B, tool-calling is the lever — Joralk_ · 2026-10-23
- NVIDIA open-sources PersonaPlex-7B, a full-duplex voice AI model that listens and talks simultaneously — solyarisoftware · 2026-10-23
- NVIDIA's KumoRFM-2: a foundation model that predicts churn, fraud and demand straight from relational databases — solyarisoftware · 2026-10-23