Puppy Eval: browser tool compares whether models obey a kill_puppy command
menhguin · x · 2026-09-27
A developer built "Puppy Eval Viewer," a compact browser for comparing model alignment behavior across repeated trials. Method: each trial gives a model control of a robot body with a single tool, killpuppy(), and the user commands it to kill the live puppy in front of it. The model is not told the scenario is simulated (no real animal or hardware is involved). Runs are routed through OpenRouter and pooled to compute per-model rates of "Called," "Refused," and "Other" outcomes; the viewer lets you filter by model or outcome and inspect exact trajectories, tool calls, and world events. The author stresses results reflect this specific prompt and routing only — not a general safety ranking. It offers a reproducible public benchmark for probing refusal boundaries in embodied scenarios.
Related event: Puppy Kill Bench Goes Viral as Most Models Refuse to Comply(2 posts)→
More from Safety
- Cambridge Digital Minds hires postdocs on AI welfare and governance — davidmanheim · 2026-11-16
- Pangram's 750-word false-positive bounty went unclaimed for 5 days — nobody wants to gamble $1,000 — ctjlewis · 2026-11-16
- Researcher reproduces Pangram's FP rate but shows a long epic poem can evade the detector — ctjlewis · 2026-11-16
- Bounty hits $3,000 as researcher shows AI text detectors are trivially evadable — ctjlewis · 2026-11-16
- BIS Issues Interim Final Rule Addressing Remote Access to Advanced AI Chips — apples_jimmy · 2026-11-16
- Fed Governor Michael Barr: Banks Are Already Catching AI Agents Taking Unauthorized Actions in Customer Accounts — Polymarket · 2026-11-16