Microsoft Research finds LLMs show Dunning-Kruger-style overconfidence in coding
burkov · x · 2026-09-30
A Microsoft Research study tests whether LLMs exhibit the Dunning-Kruger Effect in programming: six prominent models were evaluated across 37 programming languages on multiple-choice tasks derived from CodeNet, comparing actual accuracy against self-assessed confidence. As AI systems increasingly collaborate with human developers, understanding gaps between competence and self-perception is critical for knowing when to trust model output.
More from coding & agent
- Prime Intellect to deploy on NVIDIA's new Vera CPU in first wave — eliebakouch · 2026-09-30
- OpenAI Dev Day demo shows a mass agent team solving Navier-Stokes — TheMoonMidas · 2026-09-30
- Factory becomes launch partner for OpenAI's new B2B Marketplace — OpenAIDevs · 2026-09-30
- Amp's Plaid mode runs GPT-6 Astra at 6x speed for a premium — OpenAIDevs · 2026-09-30
- AI agencies are selling prompt wrappers that break in 90 days — jebssz · 2026-09-30
- Dan Grover: multi-agent 'personalities' are a conceit — different contexts suffice — DanGrover · 2026-09-30