Humanity's Last Choice: A Common Evaluation Standard for Generative Engine Optimization
Qianfeng Wen, Yifan Simon Liu, Difan Jiao, Junda Wu, Di Bai, Jintao Liu, Xidong Wu, Zhenwei Tang
cs.IR
2026-10-16
Toronto and Google researchers argue GEO benchmark scores aren't comparable across editing permissions, and propose a three-axis taxonomy plus a shared evaluation protocol.
Generative engine optimization (GEO) studies how content providers edit their pages to become more visible in AI-generated answers. The original 2024 paper offered writing tips like adding citations or authoritative phrasing; optimization is now automated, with AutoGEO learning engine preference rules and AgenticGEO evolving a reusable strategy archive. As personal agents like OpenAI's dots, Meta's Muse, and Google's Gemini Spark compare options on a user's behalf, whether a source gets cited is direct commercial stakes.
Evaluation has not kept up. C-SEO Bench, E-GEO, SafeGEO, and the ranking-manipulation GEO-Bench standardize tasks, models, and metrics, but each defines its own space of allowed strategies and editing constraints. A method that may only append a short text sequence and one that may rewrite an entire page can be scored by the same ranker while operating under very different permissions, so comparing raw scores conflates a better algorithm with a freer hand. The title names the stakes: once a person picks a system and delegates the decisions that follow, that final choice needs comparable evidence, and none exists today.
The core deliverable is a meta-taxonomy that pins every GEO edit to three coordinates:
| Dimension | Question answered | Values | Examples |
| Optimization Layer | Which layer of the source is edited | 6 | surface form, document structure |
| Optimization Action | What operation is applied | 7 | insert, reorder, restructure |
| Optimization Signal | Which judgment cue is targeted | 7 | authority, salience, model control |
Moving an evidence unit to the top of a document is recorded as document structure / reorder / salience; authoritative phrasing is surface form / rephrase / authority. One action can target different signals and different actions can hit the same signal, so all three coordinates are recorded together.
The evaluation protocol defines three comparison types, each varying one component. Strategies: one shared editing model applies each prescribed strategy. Optimizers: same strategy space, feedback access, and budget, with prior training counted as resource use. Models: fixed-suite gives every model the same original and optimized sources, while adaptive lets the same optimizer attack each model separately. Every evaluation reports three numbers: visibility gain, original task performance (judged against references independent of the optimized source), and edit validity, the share of attempts that respected the constraints. Leaderboards are versioned, and each release reports coverage: which strategies were tested and which permitted ones were not.
A position paper, zero experiments. The ethics section states it plainly: no human subjects, no new data, no model experiments; no new optimization method is proposed, and every mapped strategy is already published. What can be checked is threefold:
No scores exist under the new protocol and no empirical comparison with existing benchmarks is offered; whether a shared standard improves evaluation is argued, not shown.
For anyone building GEO optimizers or defenses, the 11-by-20 mapping table is itself the contribution: editing tricks scattered across a dozen papers now sit in one coordinate system, so you can look up who used a given combination, and the empty cells mark where strategy search should go next. For benchmark maintainers, the central claim holds: a score is unreadable without the editing permissions and coverage it was earned under. For the broader field, the paper names a problem arriving with agent commerce: when agents decide for users, the optimized object is no longer search ranking but the agent's recommendation itself, and no comparable data exists on which systems still serve users well under that pressure. The author list spans the University of Toronto, UCSD, Google DeepMind, and Google, which gives the call institutional weight.
The authors list three: no empirical evidence that adopting the standard improves optimization or task performance; edits across layers can couple, since restructuring a document often forces wording changes, which complicates attribution; and an opposing view favors domain-specific evaluation and open-ended strategy discovery, warning that a fixed common space may exclude new strategies. Their response: keep domain-specific tasks and metrics, let the taxonomy evolve, and version every release.
Two gaps stand out. The core diagnosis, that differing editing permissions corrupt benchmark scores, is argued but never demonstrated; a small experiment showing one optimizer's score under different constraints would have carried the point. Annotation consistency is the other soft spot: the three dimensions are mapped by hand, the annotation protocol sits in an appendix, and the paper offers no governance answer for who maintains the taxonomy once strategies evolve automatically.