Sort samples by loss and check the head: a practical labeling-error hack
antoine_chaffin · x · 2026-09-20
Following up on his take about noisy human annotation, Antoine Chaffin shares his go-to practice: sort samples by loss and look at the head—there's always something interesting. After finishing any eval, his first ask is to inspect failure cases and judge whether they're real failures to fix or label errors.
Related event: Qwen researcher says human-labeled data may be noisier than auto-labeling(2 posts)→
More from Research
- 200-seat UC Berkeley course on full-stack post-training, RL and applied RSI announced — karinanguyen · 2026-11-12
- NVIDIA's Déjà View: 117M looped transformer matches models 10x larger at 3D reconstruction — NVIDIAAI · 2026-11-12
- OpenAI finds models are getting harder to monitor through their chain-of-thought — xiamengzhou · 2026-11-12
- Show Reddit: ge, a Rust GGUF engine, benchmarks against llama.cpp on a 4GB RTX 3050 — Raptor2496 · 2026-11-12
- 1,500 AI-for-Science Papers in a Week Share One Question: Compared With What? — bravo_abad · 2026-11-12
- AI system proves longstanding quantum computing conjecture in three days, Lean-verified — _arohan_ · 2026-11-12