Datology: 1.5%-10.2% of public synthetic data claims are hallucinated, so it built cutoff-safe pretraining data
pratyushmaini · x · 2026-10-30
Datology AI warns that when models rephrase documents, knowledge cutoff guarantees are lost—synthetic data can introduce information its sources never had, which is catastrophic for backtesting. The company built a method to generate synthetic pretraining data with measurable, controllable knowledge cutoff guarantees.
Per its BeyondWeb Ex Ante results, 1.5%–10.2% of claims in public synthetic data are trading-relevant and hallucinated; after backtest-safety interventions, 0 trading-relevant hallucinated claims were detected across 584k evaluated.
More from Research
- AI-for-science benchmarking markets: single evals may cost months and millions, a durable niche — kenbwork · 2026-10-30
- pstack's /statechart skill ships with a mini TLC compiler that helped find real distributed-systems bugs — DavidKPiano · 2026-10-30
- Neural twins: models trained for each other, chatting in neuralese — repligate · 2026-10-30
- Self-Direction turns agent mistakes into LoRA lessons, lifting GAIA success from 38.8% to 55.8% — rohanpaul_ai · 2026-10-30
- Preprint: AlphaGenome pervasively underestimates causal variant effects, undermining fine-mapping — anshulkundaje · 2026-10-30
- One Researcher on 22 NeurIPS 2026 Papers: What Should Author Lists Mean? — qberthet · 2026-10-30