Stanford CS329X HW3: Coco agent traces plus AutoMetrics evaluators, open-sourced
EchoShao8899 · x · 2026-11-04
Details of Stanford CS329X HW3: students collect human-agent collaboration traces with a minimal fork of the open-source desktop agent Coco, then use AutoMetrics to generate automatic evaluators approximating human judgments, with an extra-credit leaderboard for the best evaluators. Models run through Tinker (Inkling models) with Persimmon for user simulation.
仓库: https://github.com/SALT-NLP/cs329xhw3
Related event: Stanford CS329X HW3: Collecting Human-AI Interaction Data with Coco(2 posts)→
More from coding & agent
- Tencent's Hy4 agent self-trained a smaller model that beat Tencent's own baseline — heyshrutimishra · 2026-11-04
- Butterfly traces multi-agent failures with graphs, 62.5% accuracy vs 53.8% for log prompting — JeremyCMorgan · 2026-11-04
- NatWest on agent safety: an 83% human-review pass rate hid a 54-point gap — AI Engineer · 2026-11-04
- Free course covers the 4 hard problems of production generative media agents — Al_Grigor · 2026-11-04
- Quota reset night: Codex back to 100% while Claude shows just 10% left — rudrank · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04