AC2 lets you train custom decision models, lifting toxicity-detection F1 from 0.482 to 0.677
rhythmrg · x · 2026-10-10
The AC2 platform now lets users train their own decision models (Jev-like), custom-fit to a use case:
- Architecture: a language-model backbone plus a small learned "decision head." The backbone processes input, question, and answer choices; the head produces scores softmaxed into a distribution over choices.
- Training: add a variable-sized decision head to any open-source model, train with Brier or cross-entropy loss; updates hit both the head and the backbone.
- Results: starting from Perplexity's pplx-decider-v1-27b and training on 50k samples from Civil Comments, toxicity-detection F1 rose from 0.482 to 0.677 on a 20k eval set.
Related event: AC2 Platform Opens Custom Decision Model Training(2 posts)→
More from Models
- Delip Rao calls out new Qwen3.5-9B-based model for benchmarking latency but not accuracy vs Jev — deliprao · 2026-10-10
- Gemini 4 Argon Launches: 77.9% on DeepSWE v1.1, Beating Claude Opus 5.5's 74.2% — dl_weekly · 2026-10-10
- Polymarket puts 28% odds on Anthropic pausing AI training this month — Polymarket · 2026-10-10
- Netlify livestream blind-tests new Anthropic, OpenAI and Mistral models on real tasks — thisiskp_ · 2026-10-10
- Google reportedly testing Gemini 4 "Carbon" internally; staff say coding "feels like Opus 5.5" — gaganghotra_ · 2026-10-10
- Max reasoning tier costs way more but scores worse on Terminal Bench 4, dev claims — weswinder · 2026-10-10