A 4M-parameter pixel transformer learns a tiny physics game for about $10
mathemagic1an · x · 2026-07-29
The author describes a toy physics simulation—collisions, wall bounces, and a goal to score in—and trains a small pixel transformer to predict the next frame.
- Model size: about 4M parameters.
- Training compute: roughly $10.
- Despite the tiny budget, the model performs surprisingly well.
- The post points to sampled hallucinations as evidence that the model has learned useful structure rather than trivial pixel copying.
Related event: 4M-Param Video Model Emerges Physics Rules as a Controllable World Model(10 posts)→
More from Research
- Nearly 2-Hour Crash Course on How LLM Benchmarking Works and Cheats — TheZachMueller · 2026-07-30
- CyberGym Level 1 is Saturated: Why the Security Industry Needs New Benchmarks — andreamichi · 2026-07-30
- Nature: AI Tool 'Raygun' Can Shrink and Supersize Proteins on Demand — Dr_Singularity · 2026-07-30
- New KSI Mechanism Externalizes Knowledge to Boost Agent Self-Improvement — yisongyue · 2026-07-30
- New Paper on Automating AI Research: LLMs Propose Ideas, Write Code, and Run Experiments — ChengleiSi · 2026-07-30
- Nearly 10% of arXiv Papers Disclose AI Usage in a Single Day — RexDouglass · 2026-07-30