Ox Alpha unmasked: reproducible fingerprint experiments and full logs open-sourced
PawelHuryn · x · 2026-08-25
Pawel Huryn shared the data and reproduction repo behind his Ox Alpha test (phuryn/experiments).- Results: OpenRouter's free stealth model stealth/ox-alpha (GLM family) found only 16 of 105 planted bugs across two repos vs 27 for Grok 4.6, deflating the hype.- Fingerprinting: the stealth-ox-alpha-fingerprint experiment identifies the model from the wire, no benchmark needed — a fixed passage costs 172 prompt tokens on ox-alpha.- Reproducibility: the repo ships exact scripts, unedited logs, and READMEs with method, sample size, and caveats, plus many other model experiments (bug-hunt-bench, fugu-ultra-vs-frontier, kimi-k3-day-one).
Related event: Stealth Model Ox Alpha Proves Mediocre at Coding, Trails Grok 4.6(3 posts)→
More from Models
- Nathan Lambert slams Meta's Muse Open License as 'trash' — natolambert · 2026-10-16
- Google launches Gemini 4 Argon to Ultra and Pro subscribers, touting physics and 3D skills — ShikharMurty · 2026-10-16
- Google gives X users early access to test Argon, not just companies — lyraxana · 2026-10-16
- Grok Voice 3.0 Tops Design Arena Speech Leaderboard, Up 7.4 Points — XFreeze · 2026-10-16
- Robotics LLMs tested on 16 tasks: GPT-6 Astra hits 17.2% success vs Fable's 9.1% — pabbeel · 2026-10-16
- Meta open-sources Muse Spark: top US open model, scores just below DeepSeek — bindureddy · 2026-10-16