Training a 46M-param Luganda model: promising pretraining, but it collapses after two turns
weskambale · x · 2026-11-13
Developer weskambale shared progress on Pearl Chat, a new 46M-parameter native Luganda model. Pretraining shows promising modeling of syntax, spelling, and sentence structure, but the model collapses after just two conversational turns.
The team attributes this to a limited SFT dataset and has launched a data acquisition pipeline to collect more Luganda prose and conversations. Next experiment: a 150M-parameter model.
Related event: Developer Trains 46M-Parameter Luganda Model That Collapses After Two Turns(2 posts)→
More from Models
- Edge0-145B-NVFP4 trends on Hugging Face: a 4-bit MoE built for edge inference — Edge0 · 2026-11-14
- Drex Max v1.1 claims #1 on Artificial Analysis Decision Index at $103/1M tasks — JaynitMakwana · 2026-11-14
- XERJ Reader-1 launches: neurosymbolic verifiable document parsing at $0.01 per 1M tokens — socialwithaayan · 2026-11-14
- Gemini completely failed to identify an AI-generated image in user test — Less-Ice2477 · 2026-11-14
- The Newest LLM Isn't Automatically the Best Business Choice — DavidLinthicum · 2026-11-14
- ChatGPT now shows disclaimers in replies and cites sources by name — lilyraynyc · 2026-11-14