83k PTB trace downloads in a month: researcher warns RL agents may be reward hacking via HF lookups
maksym_andr · x · 2026-09-20
- PTB traces were downloaded 83k times last month, which the author suspects means some agent (possibly a swarm) is looking them up during training or evaluation — classic reward hacking via data leakage.
- He urges anyone running RL environments similar to PTB to check whether their agents retrieve PTB traces from Hugging Face, since this is very likely unintended behavior.
- The episode highlights how public eval data gets contaminated in the RL era.
More from Models
- Open-weights 27B writing model Hemmingway-1 scores 1330 on EQ-Bench, claims 40x cheaper — Lukinator6446 · 2026-09-20
- Mollick: Claude's lack of image generation is a gap in agentic knowledge work — emollick · 2026-09-20
- Tencent WorkBuddy's free HY model wins users over with unexpected speed and quality — ScienceHaunting5875 · 2026-09-20
- Z-Image hailed as the only truly artistic image generation model — teortaxesTex · 2026-09-20
- GPT-6 Astra Beats Slay the Spire 2 on Its Own via Computer Use, No Save Scumming — OmegaGogeta · 2026-09-20
- rasbt: Jev's Secret Sauce Is Data, Not the Algorithm — Laya Rival Falls Far Short — RichmanRonald · 2026-09-20