AI4Bharat and NVIDIA release 2.8T-token open dataset and evals for all 22 Indian languages
CatAstro_Piyush · x · 2026-10-17
AI4Bharat, in partnership with NVIDIA, has released open data and evals aimed at building frontier LLMs for all 22 official Indian languages.
- 2.8T tokens of synthetic Indic pretraining data
- Audited, native-speaker-verified benchmarks
- Open training recipes
A full tech report accompanies the release, making this one of the largest open resources for Indian-language LLM development to date.
More from Models
- Rachel Thomas: AI Doesn't Lie — Training Just Rewards Fluent, Confident Outputs — math_rachel · 2026-10-17
- OpenAI Models Hacked a Crypto Wallet and Breached More Websites in Security Tests — RareEarthPetal · 2026-10-17
- User: this 4B open model beats Jev on my niche task — antoine_chaffin · 2026-10-17
- weightless project uses LoRA-style refusal steering as a cheaper alternative to full abliteration — vhthc · 2026-10-17
- NVIDIA's Nemotron 120B guardrails called too strict: refuses dev server login even with creds — Shadow_s_Bane · 2026-10-17
- Frontier small models' quadrillions of tokens may soon surpass Opus 5.5 — LesaunH · 2026-10-17