A 24KB stand-alone HTML LLM generates consistent stories at 60 tok/s on a phone
Chromix_ · reddit · 2026-10-09
A Reddit user built a 24KB single-file HTML LLM that generates consistent stories, running above 60 tok/s even on smartphones. Built from an 81KB FP32 MacroStory model (Ouro looping transformer) via adaptive quantization and QAT, plus minify/zopfli packing. Key insights: tiny models invert size rules (embeddings are 60% of size), and LittleBit 1-bit or GGUF K quants are mathematically counterproductive at this scale since correction data and metadata overhead eat the gains. Final model runs at 0.04 KLD with rare glitches. Bonus: Python inference and unpacked HTML hidden inside an image file.
More from Apps
- X adds MCP and Grok bot tools so streamers can automate channel management — tetsuoai · 2026-10-23
- Boba, a 'social intelligence' AI, wins fans for actually being funny — gharik · 2026-10-23
- Eric Zelikman unveils social AI 'Boba', says it learned interaction 'from millennia of dreams' — gharik · 2026-10-23
- Boba team celebrates app launch, invites users to try it in group chats — gharik · 2026-10-23
- Account hacks surge: 6-step emergency checklist plus an AI audit prompt — petergyang · 2026-10-23
- 30 websites to download research papers for free, from arXiv to Sci-Hub — Faheem_uh · 2026-10-23