Course HW3 has students simulate multi-turn human-agent interaction data and derive eval metrics
alexisjross · x · 2026-11-05
Echo Shao shared HW3 of a human-AI interaction course: students hands-on collect multi-turn human-agent interaction data through simulation and derive high-fidelity metrics for unverifiable tasks. The Persimmon model from Sandbar is provided free so students have better user simulators — a concrete example of using LLM-simulated users for agent evaluation.
More from coding & agent
- Small models ace payout fraud review in Kill Bill — same answers, 106x cost gap vs Claude — m4rkGS · 2026-11-05
- Open-source doc UI kit Extend UI nears 2M monthly npm downloads, used by major AI startups — ycombinator · 2026-11-05
- Community fork ports QuixiDesk AI desktop app to Linux with 100+ patches, Arch and signed Flatpak builds — QuixiAI · 2026-11-05
- Notion gives ChatGPT, Claude and other agents one shared memory and skills layer — nbaschez · 2026-11-05
- Claude helps owner jailbreak NordicTrack treadmill, dodges $39/month iFit subscription for Netflix — peterjliu · 2026-11-05
- SammHo: one product brief feeds launch, changelog, marketing, and support content — SammHo · 2026-11-05