Raccoons and Dumpsters: A Metaphor for Local AI Inference and MoE

GrayRoberts · reddit · 2026-08-13

The author uses a fable about raccoons and dumpsters to illustrate the evolution of AI models and hardware:

Model Evolution: Engineers initially ran virtual "raccoons" (early AI models) in "gamer dumpsters" (consumer GPUs). As capabilities improved, they trained super raccoons in trench coats (Mixture of Experts architecture), which required massive data center dumpsters to run.
Local Inference Trend: Due to high cloud rental costs, people tried extracting specific "raccoons" to run locally, but swapping models for different tasks was cumbersome.
The Apple Breakthrough: The "fruit dumpster" (Apple Silicon) changed the game by quickly loading specific experts on-demand, making it efficient to run large models on smaller local devices.
The Future: Will this efficient local system challenge massive cloud dumpster farms, and how will major hardware manufacturers respond?

More from Infra