Notion launches Knowledge Board: live eval of AI on real tasks shows high cost variance
thesaraharminta · x · 2026-08-14
Notion today launched Knowledge Board, a live evaluation of how AI models handle real-world knowledge work tasks like meeting follow-ups, support tickets, and sales pipeline updates. Built on live anonymized traffic and scored by an ensemble judge developed with researchers from Anthropic and OpenAI, it reveals high variance in cost per task and that some fastest models are also highly capable. The right model depends on the work. Not a traditional benchmark, it avoids leaderboards and shows confidence intervals.
Related event: Notion Launches Real-Traffic Model Evaluation Board(4 posts)→
More from Apps
- I benchmarked Hark, Instinct and Muse on 6 tasks — only 2 points separated them — AgentWorkinc · 2026-10-16
- Only 4.5% of US consumers pay for AI, but the top 1% of spenders drive 19.5% of revenue — dumpshoot · 2026-10-16
- Australian e-commerce brand AI bots look identical, likely from one vendor — gaganghotra_ · 2026-10-16
- Small business owner gives his AI assistant Dot its own phone number to handle clients — Early-Protection2386 · 2026-10-16
- Obsidian plugin Qiaomu AI RSS adds text-to-speech with karaoke highlighting — vista8 · 2026-10-16
- Solo seller builds voice-driven fulfillment center on Shopify, shipped 150 orders — RachelVT42 · 2026-10-16