Finance Practitioner Defends Role-Play Prompts: Benchmark Saturation Isn't ASI
nirmal_dist · x · 2026-10-25
nirmaldist defends the old role-play prompt standard ("you're a senior associate at Apollo..."). He argues it works because finance email norms have near-universal consensus, so a model failing there reflects weak alignment in finance. He pushes back on the idea that fully specified standards would saturate benchmarks: if intelligence were only strict instruction-following, that would be ASI—but expert value lies in judging which norms are actually accepted, since most industries lack a written playbook like GAAP.
More from AGI Musings
- Ex-OpenAI researcher: the 'attacker diffusion lag' shielding us from AI misuse may close fast — joshua_saxe · 2026-10-25
- Samotsvety forecasters put max unemployment by 2030 at ~9.7% aggregate (7-9% typical) — NathanpmYoung · 2026-10-25
- Median Americans Think the Typical Corporation Makes 36% Net Profit — NathanpmYoung · 2026-10-25
- Sam Altman: 'I no longer think we're gonna be able to just patch all the bugs' — GarrisonLovely · 2026-10-25
- Max AI-pilled dev: letting agents work made me treasure real-life time more — adityaag · 2026-10-25
- Burkov: Chrome ruled the human web, but Google has no answer to the agentic harness — burkov · 2026-10-25