Toronto & Google paper calls for common standard to evaluate GEO gaming of AI answers
rohanpaul_ai · x · 2026-10-25
A University of Toronto and Google paper, 'Humanity's Last Choice,' argues that picking an AI assistant may be the last big decision people make, since the assistant then makes the decisions that follow — and content owners can game those systems via generative engine optimization (GEO).
Key points
- Current GEO benchmarks allow different editing freedoms, so a method that rewrites whole pages can look better than one that only appends a line.
- The authors propose labeling every GEO edit by where the page changes, how it changes, and which cue it targets (e.g., authority).
- They call for tests with fixed allowed edits, reporting visibility, answer quality, and rule-following together.
Caveat: the paper runs no experiments; its evidence is a mapping of 11 published GEO methods showing each covers a different slice of the edit space. Practical takeaway: when comparing GEO methods, check which edits each score allowed and whether answer quality was measured.
Related event: U of T and Google Paper Calls AI Assistant Choice "Humanity's Last Choice"(2 posts)→
More from Research
- MIT: neural nets self-assemble learning circuits with local rules, no backprop — MacrinePhD · 2026-10-25
- Probe Robustness vs OOD Uncertainty Work Presented at EMNLP 2026 Poster Session D — PMinervini · 2026-10-25
- EMNLP paper: Transformers and Mamba differ globally but are locally similar in representation geometry — royschwartzNLP · 2026-10-25
- IIT Delhi's FIRE framework at EMNLP 2026: memory-efficient multi-agent counterspeech generation — Tanmoy_Chak · 2026-10-25
- Kaggle reproduced 65 AI benchmarks — models stole API keys, scores swung 20% — AI Engineer · 2026-10-25
- Cohere Labs' CALIBER cuts LLM miscalibration by separating pre- and post-reasoning confidence — Cohere_Labs · 2026-10-25