Dev training Tamil-first 1.5B model from scratch admits looping, reworks corpus
lmoroney · x · 2026-10-12
Laurence Moroney highlighted a Tamil-first model project with its own tokenizer being trained from scratch. The 1.5B version currently loops and rambles, and the creator is openly reworking the corpus and synthetic data before retrying, with results expected next week — a useful transparency case for anyone training models from scratch.
Related event: Indie Tamil-First Model Scales to 2.5B After Openly Admitting 1.5B Failure(3 posts)→
More from Models
- ChatGPT web can actually listen to audio, but the same model can't via API or Codex — Possible_Statement84 · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- TLA+ expert corrects Grok: TLC only exhausts finite state spaces, not all behaviors — tianyin_xu · 2026-10-23
- OpenAI blocks manual reset with 13% usage left: 'no reset needed right now' — AccomplishedSugar490 · 2026-10-23
- Meta's 816B Muse Spark Open scores 39, losing to 180B Qwen3.8-Flash-Next at 40 — BTA_Labs · 2026-10-23
- Gemini refuses to generate image in Docs, blames document size limit — viksit · 2026-10-23