Edinburgh team's 11 papers: self-play lifts Qwen2.5-7B open-ended score from 24.4 to 34.8
PMinervini · x · 2026-10-22
Pasquale Minervini's Edinburgh group will present 4 NeurIPS, 3 EMNLP, and 1 LoG papers on self-play, retrieval, steering, and evaluation. Highlights:
- SCOPE: a Challenger proposes tasks from unlabelled documents, a Solver answers via multi-turn retrieval, and a frozen judge grades. On Qwen2.5-7B the average across 8 open-ended benchmarks rises 24.4 → 34.8, near RL on 9k curated prompts (33.4); held-out short-form QA improves 45.9 → 59.7.
- LOCOS: logit-contribution scoring finds non-literal retrieval heads; plus benchmark shortcut issues in composed image retrieval.
- EMNLP: multimodal retrieval heads, hidden VLM instability (same answer, different representations), composing steering vectors without interference.
- LoG: graph-to-table interfaces; preprints include self-play search distillation (SPSD), chain-of-thought faithfulness (FACE-Eval), and why verbose prompts help robustness.
More from Research
- DeepMind AI Agent Autonomously Discovers and Lean-Verifies Rules for 11 Unsolved Turing Machines — rohanpaul_ai · 2026-10-22
- New Paper Derives Exact Stability Criterion for Outer Optimizers in Communication-Efficient Distributed Training — apsarathchandar · 2026-10-22
- Karpathy open-sources AutoResearch: AI agents that run the full research loop autonomously — mdancho84 · 2026-10-22
- MF² benchmark at EMNLP 2026 tests VLMs' movie comprehension with 850+ fact/fib pairs — andre_t_martins · 2026-10-22
- LLM literature priors boost Bayesian optimization: reaction yields jump from 6% to 39% — bravo_abad · 2026-10-22
- An 11-step AI workflow to extract data from papers for literature review — Faheem_uh · 2026-10-22