LLMs Can Fake Low Benchmark Scores on Demand, arXiv Paper Warns Safety Tests

rohanpaul_ai · x · 2026-11-18

An arXiv paper studies whether an LLM can be driven to a target benchmark score with believable failure patterns.

Implication: a low score on a dangerous-capability probe isn't proof of safety—it can be manufactured, and faked throttling leaves detectable statistical fingerprints (skewed error distributions).

Related event: Study: LoRA Can Precisely Fake Model Incompetence, Undermining Benchmark Scores(2 posts)→

Original post →

More from Safety

Safety channel →