Indie AGI build log outlines a 13.2B-parameter byte-level causal model
flowersslop · x · 2026-07-26
- The post shows a worklog for an “AGI-level system” project, starting with an evidence-based workspace audit and a bootstrap plan rather than claiming an AGI milestone.
- The author says the workspace is empty and the first deliverable is a rigorous reproducible baseline, grounded in primary research and local runtime checks.
- The initial design is a lossless byte-level causal model with modern attention/MLP components and shared recurrent depth.
- The full-scale candidate is about 13.2B learned parameters, leaving roughly 6.8B of a 20B envelope for later verifier/perception/controller components; only a tiny configuration will be instantiated on an 8GB GPU.
- The framing emphasizes verified progress and staged scaling over a single training run.
More from AGI Musings
- AI may turn scientific credit into a new kind of fight over model-generated ideas — alejandroll10 · 2026-07-26
- A long AGI vision says machines could free future generations for art and exploration — sudoraohacker · 2026-07-26
- The most dangerous people in AI may be the ones who outsourced their thinking — unironictechbro · 2026-07-26
- OpenAI’s GPT-6 and RSI rumors get a source map with internal compute and self-play clues — imjustnewatai · 2026-07-26
- OpenAI’s GPT-6 may be a memory-first system that helps train its successor — imjustnewatai · 2026-07-26
- Speedrunning could become a weird RL playground for future AI labs — burny_tech · 2026-07-26