Fuzzy search wouldn't help detection unless the goal is benchmark cheating

hexane360 · x · 2026-11-17

A key claim in the Pangram benchmark-cheating debate: fuzzy search over corpora wouldn't help detect genuinely new text, "unless your only goal is to cheat on benchmarks." He also argues that ensembling cheap and expensive signals shouldn't cause memorization issues unless the cheap signals are chosen poorly — implying Pangram's accuracy comes from sound engineering rather than memorizing training data.

Related event: Researchers Debate Whether Pangram's Benchmark Results Reflect Cheating(6 posts)→

Original post →