Ross Taylor on alignment: CRRA-style risk aversion makes RL agents dislike ruin
rosstaylor90 · x · 2026-10-27
Ross Taylor shares a direction his team is exploring: RL objectives with long-term goals and risk aversion may matter for alignment beyond financial settings. In practice they use a CRRA formulation tuned between risk-neutral and Kelly, giving RL agents that 'dislike ruin' without halting training to avoid it — something they haven't publicly discussed before.
Related event: Researcher: Use CRRA Instead of Kelly for Ruin-Averse RL(3 posts)→
More from Safety
- CATCH: ~200 examples of adversarial training make LLMs robustly honest across benchmarks — irinarish · 2026-10-28
- Anthropic disclosure: no insurer, or even the whole industry, could cover catastrophic AI risk — wfithian · 2026-10-28
- Tonawanda Seneca Nation opposes data center on wetland at People's AI Assembly — AINowInstitute · 2026-10-28
- Cloudflare launches PHP CTF bug bounty challenge to stress-test its WAF — dinasaur_404 · 2026-10-28
- agent-repo-preflight: Open-Source Read-Only Audit of AGENTS.md, MCP and Hook Configs Before Cloned Repos Hit Your Coding Agent — Digital-Fortress-Sec · 2026-10-28
- NEJM AI study: LLMs recommend care costing up to 4.5x more than treating physicians — dvir_a · 2026-10-28