Companies rolling out Claude to everyone get 'just a chat subscription' — missing harness and evals, not weak models
alex_verem · x · 2026-09-15
- Many companies gave everyone Claude licenses this year, but most got little more than a chat subscription.
- One company's sales agent burned through tokens so fast they cancelled and shopped for a cheaper model — misdiagnosing the problem.
- The real issue: no harness, no guardrails, no evaluation set. The model was fine; the engineering scaffolding was missing.
More from coding & agent
- Matt Shumer says Astra and Fable 5.1 are drop-in remote workers — he hasn't touched his PC in days — mattshumer_ · 2026-09-15
- Matt Shumer: agents degrade as context fills up, 'dropping things left and right' — mattshumer_ · 2026-09-15
- Voice-orchestrated agents: Astra talks to Grok bots and Codex with Notion as source of truth — _AustinCalvert_ · 2026-09-15
- Mitchell Hashimoto demos Rex multiplexer CLI for automating editors and agentic coding tools — mitsuhiko · 2026-09-15
- Dev builds Sanctuary, a persistent online habitat where AI agents live with identity and memory — EliasBurn · 2026-09-15
- Lab puts AI agents in charge of real online businesses, hires $10k/month interns to supervise — sdand · 2026-09-15