Prompt opt vs threshold tuning: cut thresholds first, then optimize prompts
dbreunig · x · 2026-10-24
dbreunig shares practical advice on benchmarking decision models: your benchmark is meaningless unless you're adjusting cuts and thresholds.
His tested workflow:
- Tune cuts first: insanely cheap and fast — in his tests, GEPA prompt optimization alone never beat it
- Then apply prompt optimization / few-shot to maximize or minimize the distribution as desired
He adds that given the price of fine-tuning these models, prompt optimization may hold long-term alpha.
Related event: Tuning thresholds beats prompt optimization for decision models(2 posts)→
More from coding & agent
- Nat Eliason builds SlopEraser, a Chrome extension that turns highlighted text rewrites into batch agent prompts — nateliason · 2026-10-24
- Claude Code helped uncover 14 speaker-muted AirTags hidden across his family's cars and bags — luisdans · 2026-10-24
- Should an AI call agent keep running when one tool goes down? A degraded-mode proposal — SubtleInterval_4449 · 2026-10-24
- Claude Desktop blocks local models; users bypass it with a loopback proxy trick — pneuny · 2026-10-24
- OpenAI ships Automations in Agents API for scheduled agent tasks — OpenAIDevs · 2026-10-24
- Arca runs its financial services business on long-horizon agents, not with them — AI Engineer · 2026-10-24