"Your LLM evaluation is lying to you": temp-0 Qwen3-235B gives 80 different answers in 1000 runs
irombie · x · 2026-10-30
Irem Ergun's write-up of her AI Connect Istanbul talk catalogs four ways LLM evals mislead you, with sources and runnable code (--mock mode, no API key). Core thesis: anyone can ship an LLM feature today, but few can prove it actually works.
Highlights:
- Models are distributions, not functions: Horace He / Thinking Machines Lab sent Qwen3-235B the same prompt 1,000 times at temperature 0 and got 80 distinct answers — identical for the first 102 tokens, then 992 said "Queens, New York" vs 8 "New York City" at token 103. Batch-invariant kernels made all 1,000 identical, proving the inference server, not sampling, caused the divergence.
- Format-only perturbations swing accuracy: Sclar et al. changed meaningless formatting details (separators, whitespace, casing) and LLaMA-2-13B accuracy moved by up to 76 points; typical spread across 50+ tasks was 10 points.
- Turkish gotcha: "İSTANBUL".lower() ≠ "istanbul" in Python, so exact-match scorers mark correct Turkish answers wrong.
She ships judgeflip.py so you can test whether your own judge flips.
More from coding & agent
- Nova v1 'Flash' launches as an AI engineering teammate aiming to replace the IDE — Gauri_the_great · 2026-10-30
- Malicious Claude Code extension with SSH backdoor hit Cursor's marketplace, up to 5,000 users — cyb3rops · 2026-10-30
- Can You Pass This 5-Question PyTorch Interview Speedrun? eval, zero_grad, detach and More — cneuralnetwork · 2026-10-30
- Full AI Short Film Workflow: Character/Props/Location Sheets + Seedance Reference Mode — socialwithaayan · 2026-10-30
- David Khourshid's top prompting trick: append 'Ask Fable what it thinks' — DavidKPiano · 2026-10-30
- Harness launches E2E-encrypted orchestration for AI coding agents across your machines — dee_hw · 2026-10-30