Federal Register caught secretly testing Qwen3 search with a mostly empty index

96Stats · x · 2026-09-17

A user with AI audit experience discovered that the US Federal Register website is quietly testing a Qwen3-powered search:

Chrome DevTools revealed internal search type IDs in the form: 3=Semantic, 6=Keyword, 306=Hybrid Qwen3, 914=Qwen3.
The Qwen option later vanished from the public page and source, but passing conditions[searchtypeid]=306 to the public Federal Register API still worked — meaning testing continues in the backend.

He ran it like a formal audit: for fixed publication dates he pulled every Federal Register document, ran unrelated queries through Qwen and keyword search, compared document IDs, unioned results, fetched metadata, and repeated across dates. The Qwen index turned out to be severely incomplete — e.g. on Nov 20, 2025, of 143 published documents, a financial query returned just one unrelated document about "Waters of the United States".

Earlier in the thread: documents synced after 3:55pm on Sept 15 were almost all findable by Qwen while earlier ones were not, and the index grew incrementally day over day — suggesting a one-day bulk load still being populated, not months of accumulation. The older Semantic search handled identical queries fine, so Qwen's irrelevant answers stem from simply not having most documents. Author concludes this was an internal test never meant to be public.

Original post →

Federal Register caught secretly testing Qwen3 search with a mostly empty index

More from Models