Astra audits economics replication code, finds errors overturning journal results
soumitrashukla9 · x · 2026-09-06
- Quoting @cremieuxrecueil's experiment: Google's Astra was run on a batch of academic replication packages and uncovered a huge number of coding errors.
- Most were inconsequential, but some overturn central results of papers in high-ranking journals; Astra also found many models weren't even run properly.
- Greg Brockman amplified it, calling Astra a good tool for checking scientific papers.
More from Models
- Local AI Goes Mainstream: TikTok Videos Have Ordinary People Asking to Run Qwen 27B at Home — StandardLovers · 2026-09-06
- OpenAI ships GPT-6 Astra, first model to hit Critical cyber threshold, at $10/$50 per token — btibor91 · 2026-09-06
- Instagram's AI detector mislabels real photos as 'AI Content', confusing users — emmanuelvivier · 2026-09-06
- Sam Altman reportedly apologizes for 'chaotic' GPT-6 Astra launch that locked out paying users — emmanuelvivier · 2026-09-06
- After 24 hours of testing, user calls Astra overhyped and says work usefulness peaked with Opus 4.8 — Scobleizer · 2026-09-06
- tokenbender on new model hype: trust shippers not yappers, verdict in 10-15 days — tokenbender · 2026-09-06