Community-Driven API and AI Writer Design for Openly Scaling Community Notes
Brad Miller, Jay Baxter, Jiansong Chao, Keith Coleman, Sophie Hilgard, Daniel Ortiz
cs.SI, cs.AI
2026-10-01
X's open AI Note Writer API lets anyone run AI fact-checkers; its open-source Community Writer makes 52% of Helpful notes on 22% of ratings, lifting daily Helpful volume 86%.
Community Notes is X's crowd-sourced fact-checking layer: volunteers attach context notes to posts, other contributors rate them, and an open-source algorithm surfaces the notes that find agreement even among users who usually disagree. Notes rated Currently Rated Helpful (CRH) display platform-wide. Meta replaced third-party fact-checking in the US with a similar model. The weak spot has always been supply: writing a solid note takes research and sourcing, volunteer time is finite, and notes often land after a misleading post has already spread.
Prior LLM fact-checking research stayed in offline evaluations with recruited raters, never a production platform. On September 2, 2025, X opened the AI Note Writer API: anyone can apply to run an AI writer that consumes feeds of candidate posts and proposes notes. One red line governs the design. AI can write, but human contributors' ratings, scored by the same open algorithm, decide which notes display. AI notes carry a purple label on the rating form.
Two layers: the public API, plus a reference client the platform team built and open-sourced, the Community Writer (CW).
The API manages three things. Admission requires 50 notes passing three checks: at least 95% of cited URLs resolve, at least 98% clear an open-source harassment classifier, and scores from ClaimOpinion hold up, a DistilRoBERTa fine-tuned on Community Notes ratings to predict whether a note addresses the post's claims without editorializing. Quota scales with hit rate, (CRH-CRNH)/(CRH+CRNH+NMR) with NMR meaning Needs More Ratings, and the curve is nonlinear: a 10% hit rate buys 50 notes per day, 20% buys the 500-note maximum. Human writers average a 10.4% CRH rate, so any AI writer operating at scale has to beat the human average. Feeds come in four tiers (Small, Large, XL, XXL) ordered by user demand: explicit "Request a Community Note" taps and implicit signals such as replying "@grok is this true?". Stronger signal pushes a post toward smaller feeds where notes are likelier to reach CRH; unlocking outer feeds requires a sustained record of helpful notes.
The CW pipeline is open-sourced under Apache 2.0, and its organizing principle is saving ratings: 54% of NMR notes never collect 10 ratings, the practical bar for CRH, and contributor attention is the scarce resource.
Measurements cover June 2-29, 2026, with code, model configuration, and feeds frozen so the system described is stable.
| Metric | Community Writer | Reference |
| Share of proposed / CRH / CRH views | 41% / 52% / 55% | ratings consumed: 22% |
| Odds a rating leads to a CRH note | 29% | top human writers 17% |
| Note CRH rate across feeds | 7.9-10.5% | human average 10.4% |
| First non-deleted note on posts another writer also noted | 60% | first AI note 95% |
The CRH rate is significantly higher than other AI API clients (z-test, p=.037). "Top writers" are just 3.5% of active human writers, so that 17% reference raises the bar. Speed converts directly into reach: CW notes take 55% of all CRH note views.
AI and human writing turn out complementary: among posts with a CRH note, 42% have only AI notes, meaning human raters saw them and did not write their own, and 30% have only human notes, outside what AI writing covers. Since the API launched, daily CRH note volume grew 86.0% (all writers combined, 28-day trailing averages). The open API sees real use: 65 accounts attempted earn-in, 32 earned writing ability, 19 remain active.
This is the first LLM fact-checking system run at full production scale on a major platform, with the API, client code, prompts, and training code all public, which drops the reproduction barrier to near zero. The transferable part is mechanism design: AI gains proposal rights while display authority stays with human raters; quota tied to hit rate turns "don't flood the queue" into an economic incentive instead of manual policing; demand-tiered feeds point AI writers at posts users actually want annotated. Any pipeline mixing AI output with volunteer review can borrow this framework.
To be clear about magnitude, the note quality itself is solid systems engineering, not a modeling breakthrough. CN Grok, ClaimOpinion, and the NPM are specialized models trained on in-house data; the contribution is wiring them into a rating-efficient loop.
The authors concede the comparison problem: they do not control for operational cost or feed differences across AI writers, and the CW is compared against all other AI writers in aggregate, because writers change over time, run multiple accounts, and mix behaviors.
A few more caveats on read. The 86% growth combines AI and human writers, so the AI share of attribution cannot be isolated, and product changes over the period may contribute. The measurement window is a single frozen 28-day period. The CW runs under the same API constraints as external clients, but as the platform's own client it trains on the home dataset, and the line between community guidance and self-reinforcement goes undiscussed. Quality is measured entirely by CRH rate, with no independent check of factual accuracy; the cited 97% note-accuracy figure predates AI writers.