This 20K-parameter, 81KB language model can still write coherent 300-word stories
x_Raincandy_x · reddit · 2026-10-07
Reddit user raincandy-u pushed TinyStories-style models to a new extreme: MacroStories, a language model with only 19,969 parameters (81 KB FP32), likely the smallest that can still generate stories.
Key specs:
- 32-dim hidden state, 378-token vocabulary
- One decoder block, recurrently applied 4 times with shared weights
- 50× smaller than the 1M-param TinyStories model, 3,000× smaller than AlexNet
It's not a general-purpose LM, but within its constrained story distribution it maintains 100–300 word narratives with a goal, problem, actions, and resolution. It runs fast on CPU with no GPU needed, and the author invites others to find where it breaks. Weights are open on HuggingFace.
More from Models
- 722 hard math problems solved at once as models scale to thousands — Dr_Singularity · 2026-10-07
- Qwen3.8 Flash Next GGUF benchmark: IQ3_S the sweet spot, 42.7M tokens tested — lxfater · 2026-10-07
- Developer regrets using Haiku to save tokens on a 'simple' task — geoffwolfe · 2026-10-07
- Grok Bot Impresses but Lacks Local Computer Use, Sticking to Cloud Machines — mark_k · 2026-10-07
- Developer finds xAI's Grok bot impressively useful for SEO, unlike Grok 4.5/4.7 — gaganghotra_ · 2026-10-07
- Musk says SpaceX's Grok bot will route tasks to Claude Opus 5.5, MidJourney, Suno and other top APIs — elonmusk · 2026-10-07