Dev lead running 50 deploys a day shares how to write and run tests in the vibe coding era

dotey · x · 2026-09-17

An engineering lead whose team ships 50 deploys and runs 300+ test rounds a day shares a concrete playbook for the vibe coding era.

Writing tests:

Test behavior, not implementation (e.g. 'duplicate order doesn't double-charge'); AI tends to write brittle implementation-coupled tests
BDD first: acceptance scenarios in Given/When/Then before code, so AI can't prove itself right
No mocks: real Redis/PostgreSQL, injected fakes with contract checks, fake timers for time logic
Prefer E2E, set 90% coverage but scrutinize uncovered failure paths; encode rules as lint

Running tests faster/cheaper:

Precise testing: trace which code each test touches, re-run only affected tests at file level
Tiering: core paths and changed tests on every CI run, the rest hourly
Right-size runners by CPU/IO needs; high concurrency isn't always faster
Self-hosted local runners cost notably less than GitHub-hosted

He promises a follow-up on reviewing AI-written code and links two precise-testing papers.

More from coding & agent