Open-source EvalCore snapshots AI app tests and replays them offline
Manvos · reddit · 2026-07-21
An open-source Rust tool for snapshot testing AI apps and agents in CI.
- EvalCore records both target and judge responses into a local SQLite cassette.
- Later test runs can replay those responses offline, avoiding repeated model calls.
- Test cases are stored in JSONL; targets, scorers, trials, and thresholds live in YAML.
- Targets can be HTTP endpoints or shell commands, and the runner exits with 0/1 for CI integration.
- The project is Apache-2.0 and distributed as a single Rust binary.
More from coding & agent
- Users say Codex side chats are becoming part of their real coding workflow — nicoalbanese10 · 2026-07-21
- LithosAI says Kimi K3 is frontier-level for open-weight agentic tasks — JiaZhihao · 2026-07-21
- Cursor agents rebuilt SQLite from an 835-page manual, with a 15x cost swing — Sam_Witteveen · 2026-07-21
- Postgres memory layer for agents reaches 73.6% on full LongMemEval, not a sampled subset — MycoBrainAI · 2026-07-21
- A roundup of AI agents and MCP resources, including how to evaluate agents — _jaydeepkarale · 2026-07-21
- A full course shows how to build and deploy an AI agent with OpenAI and LangChain — _jaydeepkarale · 2026-07-21