Running a 566GB MiMo-2.6-Pro on a 128GB Jetson AGX Thor at 0.646 tok/s
Oswald_Hydrabot · reddit · 2026-10-17
A Reddit user crammed the 566GB (FP8) MiMo-2.6-Pro onto a 128GB Jetson AGX Thor, hitting a blistering 0.646 tokens/sec — weights read from disk mid-inference. As a rebuttal to Dario's "with open source you can see the code, here we can't see inside the model," the author traced every decoder layer, routed expert, and SSD read, making the run fully replayable step by step. Equal parts meme and genuine inspection work.
More from Fun
- AWS unilaterally cancels Cloudflare's re:Invent booth; CEO taunts rival — irvinebroque · 2026-10-17
- Babylon Bee: Tiger Woods Unimpressed by Roadster Since He's Flipped Dozens of Cars — AICopyLab · 2026-10-17
- shadcn spots Claude doing 'retroactive edits' — striking through its earlier wrong answer — shadcn · 2026-10-17
- Creators fake early access to Tesla Roadster with Magnific-generated imagery, prompts included — techhalla · 2026-10-17
- Orange Tsai's HEXACON talk demos popping a browser from calc.exe, wowing security researchers — dyn___ · 2026-10-17
- repligate: model name choices likely correlate with Anthropic shibboleth usage — repligate · 2026-10-17