21 years of Nairaland scraped: 34M posts of Nigerian Pidgin released for research
saheedniyi_02 · x · 2026-10-20
A developer scraped 21 years of Nairaland, Nigeria's largest forum, and released the dataset free for research: 34,187,603 posts across 1.9M thread pages and 6.9 billion characters of Nigerian English, Pidgin, and code-switched conversation, with full thread structure preserved (who posted, when, quoting). Badly underrepresented in public training data. Available on Hugging Face (caleb-mnb/nairaland-conversations), pseudonymized, CC BY-NC 4.0, research-only.
More from Research
- Domingos Cites New Result Claiming RL Is 'Basically Useless' vs Supervised Learning — pmddomingos · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23