Cellular-automaton language model MICA lifts next-word top-1 from 15.25% to 21.5%, targets Raspberry Pi
Silver_Employ2617 · reddit · 2026-10-12
The author of MICA, an experimental language model built on a learned integer cellular automaton rather than a transformer, reports version updates. The engine uses a ring of 768 cells with 112 integer channels, 16 phases of learned local rules and 240 readout probes. Flame-W 0.3.6 (word-level) improved long-text prediction loss from 8.074 to 7.724 bits per word, and hit 33.20% on Tiny Theory-of-Mind (not statistically significant over 0.3.5). Ember 0.3.7 (byte-level core with word-level assistance, 32,768-word vocab) improved next-word top-1 from 15.25% to 21.50%, two-letter completion from 54.50% to 63.75%, and top-5 from 34.75% to 43.25%. Next step: evaluating on Raspberry Pi under strict memory/CPU constraints (no performance claims yet). Publicly available under a non-commercial license; the author asks for CPU-only baselines and evaluation-metric advice.
More from Research
- Meituan LongCat's QuantMLA: joint INT4 MLA KV caching with 3.59x compression at 128K — meituan-longcat · 2026-10-12
- MOTIVE teaches VLMs multi-perspective self-verification and when to rethink answers — Ziquan Zhu · 2026-10-12
- Free-threaded Python 3.14t plus fastsimd parses JSON at 65.7 GB/s on 64 cores — lemire · 2026-10-12
- NVIDIA's open-source SDM library unifies tabular foundation models — pandeyparul · 2026-10-12
- Qwen releases QwenGyre: elastic GPU scheduling speeds long-horizon agent RL training by up to 1.85x — Alibaba_Qwen · 2026-10-12
- Crowdsourced push on integer multiplication bounds stalls: κ improves only 0.7%, geometry experts sought — iskander · 2026-10-12