Self-trained 60MB quantized LLM supports 1M token disk context
Final-Data-1410 · reddit · 2026-08-24
A developer open-sourced SHADOW-250M, a self-trained 250M parameter model quantized to under 2 bits, requiring only 60MB for deployment and running at 400 tok/s on CPU. It features a unique context mechanism where recent tokens remain in fp16 cache while older history is compressed to 1 bit on disk, theoretically supporting up to 100M tokens of retrieval. The demo shows accurate retrieval of a serial number buried 50.6M tokens deep. The repo includes code, fine-tuning tools, and benchmarks.
More from Research
- LightSuite: fast cell detection and whole-brain atlas registration on a standard workstation — hisspikeness · 2026-10-15
- Pathway CSO pitches BDH as post-Transformer architecture uniting memory, reasoning, adaptation — schwentker · 2026-10-15
- AI-designed molecular scissors cut Alzheimer's proteins — but is it really AI? — gerardsans · 2026-10-15
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Google's HARNESS² evolves agent harnesses without verifiers, up to +11.5 points on WorkBuddy-Bench — dair_ai · 2026-10-15