Chollet and critics clash over image-gen eval favoring edge alignment over perceptual similarity
giffmana · x · 2026-11-06
A technical debate over an image generation evaluation. Critics argue the "convex cover blob" method looks no worse than the model on any human-perceptual metric, claiming the scoring is designed to heavily favor edge alignment over perceptual similarity — and questioning why the model can't see the scoring system. François Chollet (giffmana) responds that edge alignment is indeed a key component here, since other perceptual metrics largely measure texture and there is none in these samples, though he concedes the eval could be better.
More from Research
- SteerAct lifts robot action-model success ~10% with just 24 rollouts per task — pliang279 · 2026-11-13
- PhD thesis explores intra-turn interaction: AI interfaces beyond turn-based dialogue — OfirPress · 2026-11-13
- CMU trains memory writing with RL: Qwen3.5-4B rivals GPT-5.4 on continual learning — aviral_kumar2 · 2026-11-13
- NTU paper: decision info lives in middle layers, last six layers removable loss-free — Minh-Anh Nguyen · 2026-11-13
- Inkling 2 Small's width-4 sconv forces a tiny rolling 3-token conv cache per layer at inference — andrew_n_carr · 2026-11-13
- Open-source legal conflict detector runs on encoder models only, 133-clause contracts in under 20s — ladyofcode · 2026-11-13