BioAgent-DF: silent hardware faults can alter AI science agents' decisions invisibly
INTOSYN-AI-Labs · hf · 2026-11-11
INTOSYN AI Labs (with Columbia and Microsoft) released BioAgent-DF, a framework for auditing decision provenance in biomedical research agents.
- Key finding: an agent can reach the same verdict while citing different evidence, and silent numerical hardware faults can change evidence selection and experimental decisions while aggregate metrics look unchanged.
- Method: paired reference/candidate values are captured at computation sites and replayed on healthy hardware. Across 8 SciFact batches, Qwen3.5-4B and Gemma 3 4B IT showed repeatable citation changes.
- Case study: in hERG cardiac-safety assay allocation, restoring the agent's earlier execution state plus certified scores recovered all 39 changed decisions; replacing the prediction file recovered none.
- Streaming arithmetic checks cut forward-execution-plus-verification time by 42.4% vs the FP64 baseline.
Takeaway: audits must track both the evidence an agent uses and its final answer.
More from Research
- Sakana AI × Pathway Post-Transformers lecture on AI and the brain now online — kaixhin · 2026-11-11
- Defending METR: public data can't second-guess its AI speedup findings — peterwildeford · 2026-11-11
- Adversarial RL envs for backdoor insertion vs. detection: where does it converge? — DimitrisPapail · 2026-11-11
- UniPath (NeurIPS 2026): adaptively coordinating understanding and generation in one model — jindong_wang92 · 2026-11-11
- Versor pitches real2sim2real: robotics sim needs physical dynamics, not just visuals — abhishekunique7 · 2026-11-11
- Scale AI's 1,200-run study: PPO wins at small batches, GRPO catches up at scale — heghbalz · 2026-11-11