Economists use AI to unlock 70 years of Indian district handbooks as data
paulnovosad · x · 2026-11-09
Economist Paul Novosad's team is running a crowd-sourced project to turn India's district handbooks—rich in socioeconomic content since at least 1951, down to village-level public goods and irrigated land—into usable data with AI.
Challenges: no site hosts all handbooks (they're building one, 75% collected, 1991 coverage weakest); massive state-to-state format variation requiring table parsing and cross-state schema reconciliation; and harmonizing villages/districts over time (spelling variants, boundary changes), leveraging prior work like @indiastatestory's district-change data.
They invite anyone with parsing scripts, scraped data, or missing handbooks to get in touch.
More from Research
- Georgia Tech Lunar Lab brings 3 robot papers to CoRL 2026: deformable dynamics, navigation, humanoids — fdellaert · 2026-11-10
- Catherine Olsson revisits the spec gaming master list of AI reward-hacking examples — catherineols · 2026-11-10
- Oxford releases 180-page mechanistic interpretability handbook with 51 methods and 19 notebooks — Hesamation · 2026-11-10
- Workspace Models presented at CoRL 2026, poster session at JW Marriott — rishabh16_ · 2026-11-10
- PCBRouteAgent: multimodal LLM agent routes 84/86 boards vs baseline 10/86 — Yue Zhang · 2026-11-10
- NVIDIA's CPU-ARC: reasoning caches 16–291x cheaper with frontier-level performance — nvidia · 2026-11-10