Developer trains a 1M-param LLM from scratch on a 16GB MacBook M1 with French corpus
Slight_Ad_2894 · reddit · 2026-09-05
A developer shared a project building a tiny LLM from scratch on a 16GB MacBook M1: trained a 1M-parameter neural network with a BPE tokenizer on a French corpus, aiming to understand and teach how LLMs work rather than reach SOTA quality.
The model produces grammatically decent sentences but lacks contextual understanding and can't answer questions well. The author is asking for resources to improve it — typical advice would be that the tiny scale only learns syntax, and scaling up plus fine-tuning on instruction data (or following resources like nanoGPT and Karpathy's "Let's build GPT") is the path forward.
More from Research
- Mathematician adds context on how professionals actually use models for math — littmath · 2026-09-05
- Astra beats Sol but not SOTA on hard wet-lab biology, citing scarce public data — nlarusstone · 2026-09-05
- 257 molecules in on STAT6: all to be wet-lab validated, $1,000 per qualified binder — AllThingsApx · 2026-09-05
- AI ≠ LLM: Classical ML Still King for Hardcore Science, Researcher Argues — CatAstro_Piyush · 2026-09-05
- Stanford paper shows sycophancy transfers through neutral data in 7 preference optimization methods — stanfordnlp · 2026-09-05
- Stanford's Marin 535B-A23B Open Training Run Hits 13%, Funded by Jensen Huang's Foundation — stanfordnlp · 2026-09-05