Personal benchmark 'I don't know' fine-tunes open models via Tinker to test if models know when they don't know

brandon_galang · x · 2026-11-17

Developer loganzev released The Trust Table, a personal "I don't know" benchmark built with Thinking Machines' Tinker API. He fine-tuned open-source models and developed games for them to play and learn from, exploring whether models know when they don't know the answer. He cautions it's a first-of-its-kind product/research experiment, with results specific to this game, but says he learned a lot from the process.

Original post →

More from Research

Research channel →