CRUX-in-a-box: open-source infra for weeks-long, monitored agent evaluations
sayashk · x · 2026-11-06
Sayash Kapoor announces CRUX-in-a-box, open-source infrastructure for running and monitoring weeks-long agent evaluations, built from six months of long-horizon eval work in CRUX.
Key capabilities:
- Run agents on VMs for weeks, beyond local sessions
- LLM-based security review: analyze logs/snapshots and flag risky actions for human review
- Inspect tool calls, token usage, and spend
- Set time and cost budgets, with optional custom SDK phase checks
- Repeatable VM setup with separately configured credentials
- Slack alerts and mid-run chat to give agents feedback or answer questions
Available on GitHub.
Related event: CRUX-in-a-box Open-Sources Infrastructure for Weeks-Long Agent Evaluations(3 posts)→
More from coding & agent
- Ampersand Claims 92.2% on SWE-bench Pro HARD-51 at 70% Lower Cost — niloofar_mire · 2026-11-06
- ServiceNow Brazil Release Touts 100x Gains with Autonomous Engineer — azavery · 2026-11-06
- LiteParse Integrates 1.3B MiniCPM-OCR, Beating Gemini Flash on ParseBench — llama_index · 2026-11-06
- Finix MCP Server lets AI agents create buyers, payment links, and search docs — modelcontextprotocol · 2026-11-06
- Agent Substrate ships an MCP server for fs/bash on actors with auto suspend-resume — rakyll · 2026-11-06
- Personal agents weekly: Tomo adds Shop Pay via MCP, Fo's Twilio calls span 100+ countries — heyneighbor · 2026-11-06