2026-08-25
A four-page awards doctrine: five binary gates, then ten weighted pillars. Models may organize evidence; named humans keep recognition authority.
AI-native work arrived faster than the means to judge it. Ordinary criticism assumes a human author and scores the finished surface. Here the method is part of the art: which models were orchestrated, how identity and continuity were held, whether a success is a method or a lucky draw. Traditional awards never had to measure those. The other failure mode is circular. Hand the score to a model, and the thing under evaluation grades its own exam.
This is Orchestrator Awards Working Paper Issue 003. Four pages, not peer-reviewed. Mark Sendo sits inside the institution that would confer the award, discloses that interest on page one, and notes that the text was drafted with AI assistance. The paper is not a live evaluation. It argues that three commitments have to hold together: admissibility before merit, evidence before any claim is credited, and recognition authority that stays human for good.
Eligibility and scoring are separate. A work first clears five pass/fail gates. Fail one and evaluation stops; quality cannot buy an exemption:
Only then does scoring start. Ten weighted pillars are published: Orchestration Craft 16%, Story and Intent 14%, Identity and Continuity 11%, Technical Execution 10%, Transparency and Disclosure 10%, Human Creative Control 10%, Originality and Creative Novelty 9%, Safety and Rights Discipline 8%, Workflow Evidence 8%, Field Contribution 4%. The weights are public so they can be argued with.
Disclosure is tested twice, on two different questions. As a gate it is honesty: was material AI use told in full. Incomplete disclosure is disqualifying. As a pillar it is quality: once told, how precise, versioned, and auditable is the record. In a form where method is part of the work, hiding the method is itself a failure.
The machine's role is fenced. Named, unconflicted human evaluators issue the verdict. No model decides eligibility, nomination, finalist status, or recognition. If model help is ever introduced, it may only organize information the entrant already supplied. Scope, evidence handling, auditability, and limits would be published and separately authorized first. The firewall does not move with capability.
Further rules keep the humans honest. Evaluators score independently and cannot see each other before lock, then scores are aggregated. Conflicts are disclosed and recused, and the recusal record is part of the audit trail. Verdicts can be challenged; evidence disputes and record errors have a correction path. The Pioneer Award is a board honor that skips the five gates and ten pillars, under a separate published honorary method. Naming the exception is meant to stop it from eating the rule.
Domain literacy is bounded. Evaluators are picked for competence in the relevant form. Literacy in one domain is not treated as authority in another. The instrument can be shared; the people who read it cannot be treated as universal.
There is no experiment, no benchmark, and no inter-rater number. What the document delivers is a methodology published before it is used, plus a hard non-activation clause: publication does not start scoring, submissions, nominations, juries, voting, or recognition. The doctrine line is "Publish first. Be cited second. Recognize third."
The checkable structure is small. Five gates are binary. Ten pillars sum to 100%. Disclosure appears both as a gate and as a pillar. Recognition authority is placed in the class of powers that The Custody of Intelligence, a prior piece by the same author, says must not be autonomously delegated. The argument is structural. Even a highly accurate model cannot be the party that answers for a verdict.
Anyone running AI-content contests, leaderboards, or platform review will recognize the opposite of the current cheap default: using an LLM as judge. This paper calls that default illegitimate, because generated work judging generated work breaks the accountability chain.
The portable pieces are mundane, which is why they travel. Split admission from scoring so a polished result cannot launder an undisclosed method. Demand version-locked workflow logs rather than author claims. Have humans score independently and then aggregate. For internal creative review, procurement acceptance, or open contests, those three rules are more useful than the exact pillar weights.
Read as an evaluation paper it will disappoint. Nothing here shows that the pillars separate a method that solved a hard problem from a one-off lucky sample, or that human judges agree on orchestration craft. It is closer to a draft constitution for an awards body.
The author, the Orchestrator Institute, and the Orchestrator Awards sit under Celestial Technologies LLC. The conflict is disclosed and still means the institution is writing rules for its own prize. The text is not peer-reviewed, four pages long, and denser in claims than in evidence.
The five gates and ten pillars are not specified to a reproducible rubric. What counts as complete disclosure of material AI use, and what a 16 on orchestration craft looks like, have no scale, no scored examples, and no calibration. The Pioneer Award bypasses the competitive discipline entirely, so the highest honor is the least bound. The line "models may organize, never judge" will be easy to thin out in practice; the paper states the ban and does not state how the ban would be audited.
Because nothing is activated, outsiders cannot yet check whether the method is used, or used tightly.