review-replay checks whether PR review comments were actually fixed, not just marked addressed
alejandro_such · reddit · 2026-07-26
The author built review-replay, a Go tool that verifies whether PR review comments were actually resolved instead of trusting a generic "addressed" reply.
The tool is meant for a world where humans and agents both touch the same PRs. A reviewer’s comment can be answered by Copilot, Claude Code, or Cursor, but the reply does not prove the concern was fixed. Review-replay reads the PR conversation and the code at HEAD, then classifies each comment as:
- addressed
- partial
- pending
- needs-discussion
It uses deterministic rules first, then calls an LLM only for ambiguous cases to keep token cost low. It can run as:
- a CLI command with --check
- a GitHub Action that blocks merge on unresolved feedback
- an agent skill that self-checks after edits
It supports OpenAI-compatible providers, local models via Ollama or LM Studio, and includes an eval harness to measure classification accuracy.
Repo: alejandroSuch/review-replay
More from coding & agent
- Inference providers may need smarter cache windows than a fixed five minutes — lucasmeijer · 2026-07-26
- A coding agent used remote browser control to build a campground search site — yacineMTB · 2026-07-26
- How one developer stopped being vague with an AI coding agent — rseroter · 2026-07-26
- Karpathy says AI coding is about knowing when to delete the worktree and say no — lemire · 2026-07-26
- Productive Engineer Shares AI Coding Workflow: Never Reads Code, Relies on Agent Plans — danshipper · 2026-07-26
- Creator says Claude Code helped him earn over $250,000 by shipping member software — EXM7777 · 2026-07-26