scaling01 says the NCRI/NCKI benchmark contamination loop he flagged now appears in system cards
scaling01 · x · 2026-11-10
KOL scaling01 recounts that after he found and amplified Neel Nanda's post on NCRI/NCKI and kept calling out the "dirty loopers" contaminating benchmarks, every major system card now reports the issue. His self-deprecating take: "not saying I had any impact, just that I was early again."
Related event: KOL Claims Credit for Getting "Dirty Loopers" Into Model System Cards(2 posts)→
More from Models
- Claude Fable 5.2 tops VIALS-AA science visual reasoning benchmark at 67% with heavy tool use — ArtificialAnlys · 2026-11-10
- Dev builds a mobile game with Fable 5.2 as a Claude Artifact in under an hour — ChrisGPT · 2026-11-10
- SpaceX says Grok detects aerospace foreign object debris with 99.9% accuracy vs humans' 94% — elonmusk · 2026-11-10
- Fable 5.2's first ASCII backrooms art is so good people suspect it's programmatic — repligate · 2026-11-10
- Community pushes AI labs to publish system cards as machine-readable jsonl data — scaling01 · 2026-11-10
- ARLoop: fully-open 1.2B looped transformer beats Huginn-3.5B and Qwen3.5-2B on all 8 benchmarks — hhsun1 · 2026-11-10