Google's SynthCorp generates verifiable data-agent benchmarks with up to 1,000 tables per estate
dair_ai · x · 2026-11-13
Google presents SynthCorp, a platform for building data-agent benchmarks that saturate less. It simulates routine and disruptive business events on a virtual calendar, materializing consistent relational tables, a document store, and a data catalog. Every operation lands in a simulation log, so gold answers are derived and verified without manual annotation. The first release ships seven data estates (up to 1,000 tables) and 256 tasks across five domains; multi-event composite tasks still trip state-of-the-art agents. Open-sourced for the community.
More from Research
- NextLat releases final paper revision: multi-token belief states, Mamba gains — JohnCLangford · 2026-11-13
- Genomics x AI community blog launches with AlphaGenome tutorials and more — anshulkundaje · 2026-11-13
- CVPR'27 paper deadline 4 days away: AI conference countdown — AlexTensor · 2026-11-13
- Each 10x in compute buys roughly 10x longer AI time horizons, elasticity near 1 — ben_j_todd · 2026-11-13
- NeuroAI work clarifies CNN vs transformer brain models — transformers win in interesting ways — dyamins · 2026-11-13
- OpenAI's $0.11 model beats the priciest LLMs in new Slack agent benchmark — heyshrutimishra · 2026-11-13