HF researcher: auditing online eval biases turned into the biggest section of my BP
antoine_chaffin · x · 2026-11-03
Hugging Face researcher Antoine Chaffin notes that while writing a business plan for a decider model, the "biases" section has become one of the largest parts. In a quoted tweet he says he plans to publish a breakdown of all the biases and broken evals he found in online evaluations — a humbling exercise that reveals what a model actually learns, how it really improves, and how everyone keeps falling into the same traps.
More from Research
- Intology's Locus autonomously adapts Boltz-2 and frontier scientific models to proprietary data, claiming SOTA-beating molecular design — anshulkundaje · 2026-11-04
- Re-testing 50 cited LLM findings: most fade within 15 months — cephaloform · 2026-11-04
- Simon Willison replicates GPT-6 Astra weight-interpretation feat with Qwen 3.8 27B in 45 minutes — cephaloform · 2026-11-04
- New XAI study links latent-space causality to real flow structures via SHAP — ricardovinuesa · 2026-11-04
- Median human turn-taking pause is -120ms, so voice AI's 500ms brag still misses how people talk — alexisgallagher · 2026-11-04
- ParallACT: when every dev has their own coding agent, coordination still breaks down — EliasEskin · 2026-11-04