Trained to be wrong 98% of the time at 96% confidence: meet Bev, the anti-model test fixture

ricyoung · reddit · 2026-10-07

ricyoung fine-tuned Bev on Qwen3.5-9B to deliberately pick the worst answer while staying confident.

• Results: on 324 held-out decisions she's right 1.9% of the time at 96% average confidence (1.4% when ≥90% sure). "ADI: Artificial Drunk Intelligence"
• Key training lesson: flipping labels on the base model failed twice (coin-flip behavior after 2 GPU-hours); starting from Bespoke's Nimble adapter — which already knows the answers — and teaching it to invert took 51 minutes to reach 97% wrong. A model must know the right answer to be reliably wrong
• Purpose: a control case for "act automatically if the model is ≥90% sure" pipelines — if your system doesn't flag her, it isn't checking what you think it is
• Details: works with Ollama's new decision endpoint (/v1/systemone), 3 GGUF quants, Apache-2.0, 3h38m on one 4090; Q4KM flips 20/324 answers vs bf16 so Q80 is default
• Try in browser (HF Space) or ollama run richardyoung/bev; code and training logs open-sourced

Original post →

More from Fun

• Grok's quirk: it says 'No.' then argues your point better than you did — gandamu_ml · 2026-10-07
• Mistral turns the Le Chaton fat cat meme into a real thing — _AndrewZhao · 2026-10-07
• Audio-reactive 3D director demo: drive procedural rigs live from your phone — DimitriDeJonghe · 2026-10-07
• Dev Recreates All Adobe Apps with Opus 5.5, Ports Them to Rust, Open Sources — SumitGup · 2026-10-07
• Jordi Pons launches interactive AI music game 'A Chicken Dies' — jordiponsdotme · 2026-10-07
• Swap "AI" for "Super Intelligence" and Every AI Flop Headline Gets Funnier — Roodut · 2026-10-07

Fun channel →