Kolab benchmark: 115+ hours of human-AI collaboration videos across 40 occupations
windx0303 · x · 2026-10-28
Shirley Wu's team introduces Kolab, a benchmark built from 115+ hours of human-agent collaboration videos spanning 40 occupations and 50 professionals in long-horizon workflows. The core argument: rather than benchmarking whether frontier models can fully automate tasks, we should study how they collaborate with human professionals. Dongyeop Kang amplified it.
Related event: Kolab Benchmark Tests Human-AI Collaboration with 115 Hours of Real Video(2 posts)→
More from Research
- Screening 307 AI-for-science papers in one week: the test that matters is out-of-distribution — bravo_abad · 2026-10-28
- Why K=4 beats K=32 in group-sampled RL training: filtering easy prompts faster — gowthami_s · 2026-10-28
- What 'hard prompts' look like in vision: rare concept combos like armadillos in berets — gowthami_s · 2026-10-28
- Robotics may need to go beyond language: demos as a test-time steering modality — Ed__Johns · 2026-10-28
- VybBench: open benchmark scores image models' design fidelity with numbers, not votes — thetripathi58 · 2026-10-28
- Vision Transformers emergently segment objects—new work explains how via Gestalt psychology — aryaman2020 · 2026-10-28