Sedielem: Model-Based Evals Stay Biased, and Sliced Wasserstein Is Everywhere Now
sedielem · x · 2026-10-11
Researcher Sedielem offers two observations on model-based evaluation metrics:
- While more robust than prior model-based approaches, these metrics remain constrained by the chosen judge model's biases—e.g., evaluating modern LLM outputs with GPT-2 as the critic would still go badly.
- The sliced Wasserstein distance is rapidly becoming a tool every ML practitioner should have in their toolbox; it keeps popping up everywhere.
More from Research
- Embedding arithmetic demo: cartoon − animation − comic = diagram — mayfer · 2026-10-23
- Omnii Genome: Single Generalist Model Outperforms AlphaGenome Across All Benchmarks — JosephJacks_ · 2026-10-23
- Domingos Cites New Result Claiming RL Is 'Basically Useless' vs Supervised Learning — pmddomingos · 2026-10-23
- Word Arithmetic on Embeddings: "Bathroom Minus Toilet Equals Kitchen" Beats SAE Probing — mayfer · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23