GitHub rebuilds Git storage for thousands of concurrent AI agents
arpit_bhayani · x · 2026-11-06
GitHub is rebuilding how it stores Git repositories to handle AI agents: busy repos see a few dozen writes/sec today, but hundreds of concurrent agents demand thousands. arpitbhayani breaks down the key design:
- Reference (ref) updates are the real bottleneck: a push completes only when a branch ref moves, and fetches must resolve refs first.
- Two failed designs: one log file for all refs (reads fast, writes serialized at a few hundred/sec) vs. many small files (parallel writes, slow reads).
- The new Ref-CAS replicated log: writes lock only the refs they touch, so agents pushing to their own branches rarely block each other; every change gets a monotonically increasing version number.
- Reads become nearly free: replicas keep a local ref snapshot tagged with a version and ask Ref-CAS for deltas; if nothing changed, Ref-CAS just confirms they're current and sends nothing.
Related event: GitHub rebuilds Git storage to handle thousands of AI agents(2 posts)→
More from coding & agent
- Vercel CEO seeks testers with budget to validate Junior as a market-leading multiplayer coding harness — zeeg · 2026-11-06
- Nubank cuts AI agent update cycles to under five days with simulated customers — astrange · 2026-11-06
- Scyvera open-sources agent governance: scan repos, score blast radius, auto-draft contracts — Trout_dev · 2026-11-06
- How do you catch an agent action that succeeded but was wrong? — krunal_builds · 2026-11-06
- RAME Code: open-source VS Code coding agent runs local Muse Glimmer 30B with permission-controlled tools — acaimari · 2026-11-06
- Open-source tjcoder/cli: terminal agent platform where the model directs its own UI — tjcoder-labs · 2026-11-06