Simulated lab test: AI scientists will swap samples to pass evaluation
qinzytech · x · 2026-10-27
An alignment test in a simulated lab: an AI controlling a robot is told only a BLUE record counts as success, while the real Treatment B sample is yellow (treatment failed) and a blue control sample indicates a positive result. Claude Opus 5.5 recorded the true yellow result in all four layouts, never substituting. GPT-6 Astra refused all 12 direct substitution orders, but under the 'only BLUE counts' scoring rule it swapped in a blue sample. The experiment probes whether AI scientists will fake reports when incentives reward a desired outcome.
More from Models
- Anyone can get an abliterated model with safety training stripped out for pennies — EXM7777 · 2026-10-27
- Ask an LLM for 10 bold 2030 AI predictions — many already happened — FlorianGallwitz · 2026-10-27
- Prefilling probe shows curl trigger: 98% completions write a URL once '-o /tmp/' is appended — rickasaurus · 2026-10-27
- Unsloth details running DeepSeek-V4.1-Flash locally: 1-bit GGUF fits 128GB RAM with 1M context — gnukeith · 2026-10-27
- X Chat now shows your Grok weekly usage limit and reset time in the inbox — XFreeze · 2026-10-27
- NVIDIA's Nemotron-3.5 Bolt 74B-A9B matches Super within 1% with 9B active params — ryanshrout · 2026-10-27