Meta Muse hands-on: fails as tasks pile up, "scales to millions but not personally"
deliprao · x · 2026-09-19
Delip Rao ran controlled experiments on Meta's Muse agent and concluded it "scales to millions of users but does not scale personally":
- Incrementally adding complexity showed that the more cron jobs it tracks, the more slip-ups occur, until it stops following earlier important instructions;
- Beyond typical model/harness failure modes, he suspects silent failures invisible to users, like rising cron job failure rates as user count scales;
- Muse repeatedly asked for information already in its context (model issue) or in longer history (harness issue).
His take: Muse is well-designed with good UX but remains a reactive agent requiring user-initiated push, not a personalized ambient intelligence companion—perhaps not Meta's goal anyway. He draws a broader lesson about owning your own intelligence infrastructure with dedicated compute. He notes it's an n=1 observation, but argues vibe evals matter for AI products.
More from coding & agent
- Butterfly traces multi-agent failures with graphs, 62.5% accuracy vs 53.8% for log prompting — JeremyCMorgan · 2026-11-04
- Free course covers the 4 hard problems of production generative media agents — Al_Grigor · 2026-11-04
- Quota reset night: Codex back to 100% while Claude shows just 10% left — rudrank · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04
- Founder of 50-person, 12-country team says he talks to AI agents more than humans — Scobleizer · 2026-11-04