Custom 250M LLM: 60MB Deploy, 100M-Token Disk Context Retrieval
Final-Data-1410 · reddit · 2026-08-22
Trained a 250M parameter model from scratch on 30B tokens, quantized to <2 bits for a 60MB deployment. It runs at 400 tok/s on CPU. Its key feature is a long context mechanism where tokens older than 2048 are compressed to 1 bit and stored on disk (320 bytes/token), supporting retrieval from up to 100M tokens of history. The vocabulary uses fixed 512-bit codes with zero trained parameters. The model successfully retrieved a serial number from 50.6M tokens deep. Code and weights are open-sourced.
More from coding & agent
- Shipped community-designed error handling for MCP server; TS frontend analysis achieves 96.6% route matching — emahmoudnabil · 2026-08-22
- Is building full-stack websites with Codex or Claude viable? — Bravia_Kafkaa · 2026-08-22
- James Long: Sandboxes are about unbounded resources, not security — Vjeux · 2026-08-22
- Anthropic engineer demos building 5 AI assistants in one afternoon — Roger_M_Taylor · 2026-08-22
- Browser fingerprint incoherence, not models, gets web agents blocked — thalissonvs · 2026-08-22
- Does a 10-page prompt really lower the barrier for AI agents? — DaniloJRezende · 2026-08-22