UK AISI Adopts EvalEval's Evaluation Cards to Make Benchmark Results Reproducible
ben_burtenshaw · x · 2026-09-23
A Hugging Face article announces that the UK AI Security Institute (AISI) is now using the EvalEval Coalition's infrastructure to openly share evaluation results, supporting reproducible and verifiable evaluation science.
- Context: as AI deployment accelerates, evals are key evidence of model performance, but results are scattered across formats and platforms, often without enough detail to reproduce them — and re-running evals can be prohibitively expensive.
- Solution: EvalEval's shared reporting schema "Every Eval Ever (EEE)" plus an open platform of Evaluation Cards that capture results, configs, and protocol context in one structure.
- The collaboration began at a joint NeurIPS 2025 workshop; AISI feedback shaped the EEE schema. AISI also built OptStop for eval efficiency and HiBayES for statistical rigor.
- Key point: reproducible evals need more than a leaderboard score — benchmarks are measurements in context, not facts.
Related event: UK AISI Partners with EvalEval to Open Up AI Evaluations via Eval Cards(5 posts)→
More from Safety
- Senate Bill Would Force Big Platforms to Accept AI Agents, Raising Auth Privacy Questions — Prof_CarlaReyes · 2026-11-12
- United Airlines Bans AI Agents From Booking Flights Over Data and Legal Risks — rohanpaul_ai · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Focused task framing boosts agent boundary testing by ~60 points; VM escape still fails — perplexity_ai · 2026-11-12
- Perplexity red-teams its SPACE VM sandbox: CTF prompts mislead agent security testing — perplexity_ai · 2026-11-12
- Narayanan: blanket bans on teen AI companions are bad policy; incentivize pro-social design instead — random_walker · 2026-11-12