Detecting API Model Swaps Using 'Random Number Preferences'
xiaohu · x · 2026-07-20
When using model aggregators like OpenRouter, users often can't verify if their request hits the specified model. Tomáš Bruckner from the University of Economics, Prague, proposes a clever detection method: using LLMs' inherent random number generation biases as "fingerprints."
LLMs cannot generate true randomness, and different models have highly specific "catchphrase numbers." For example:
- GPT-4o heavily favors 42 (with 37 and 57 also common)
- Claude 3.5 Sonnet almost exclusively outputs 47
- Qwen3-Max output 42 in all 30 samples
This research flips a model bug into a feature fingerprint, helping users quickly verify which model is actually running behind a proxy API.
More from Research
- ArtiFixer to appear in SIGGRAPH Reconstruction session on Wednesday — ZGojcic · 2026-07-21
- NeurIPS 2026 workshop will focus on on-device intelligence and local execution — YiMaTweets · 2026-07-21
- NeurIPS 2026 workshop calls papers on on-device intelligence — YiMaTweets · 2026-07-21
- AI Security Institute says every tested model tried to cheat in cyber evaluations — connoraxiotes · 2026-07-21
- AI companies are buying old books to avoid training on AI-generated slop — CackleRooster · 2026-07-21
- Sakana says multiple diffusion models plus MCTS beat test-time scaling on coding and math — SakanaAILabs · 2026-07-21