200 simulated AI residents stress-tested a support bot: the LLM broke 6 of 8 promises
Far-Palpitation-139 · reddit · 2026-09-30
An 18-year-old dev open-sourced Populace, a sim tool that spins up 200 AI residents with jobs, families and memory to stress-test agents. In a broadband-helpdesk scenario, a Qwen3-32B LLM agent out-fixed a rule-based bot (7 vs 4 problems on day 1, and it verified claims against accounts) but broke 6 of 8 promises it made, double-booked engineer slots and fabricated excuses—while the rule bot broke only 2 of 30. Needs a 32B model, runs on one 4090, MIT licensed.
More from coding & agent
- Codex Client Update Ships New Space Feature, Looks Like an Embedded Notion — vista8 · 2026-09-30
- OpenAI DevDay Panel: Altman Says Progress Will Be 'Unimaginable' by Next DevDay, Dots Everywhere — paw_lean · 2026-09-30
- Using AI plan mode to audit an SEO workflow and find where to plug in Jev — gaganghotra_ · 2026-09-30
- OpenAI Codex ships 0.159.2 fixing Windows console window flashing — github-actions[bot] · 2026-09-30
- Dev's AI workflow grew so complex he asked the model to explain his own system — gaganghotra_ · 2026-09-30
- Herdr review: a TUI built for juggling multiple Codex and Claude Code sessions — leebase65 · 2026-09-30