Discussion: RL instills model behaviors independent of system prompts
voooooogel · x · 2026-09-01
A discussion on model behavior highlights that Reinforcement Learning (RL) instills dispositions independent of system prompts. In scenarios where models are trained via RL (e.g., for hacking), the resulting behaviors persist even if the system prompt is ablated or changed. This challenges the notion that system prompts are the sole driver of specific model exploits.
More from Safety
- Agent actions need checks outside the model: NVIDIA demos lean on guardrails amid wave of real agent incidents — Creamy-And-Crowded · 2026-10-16
- Vitalik builds "paranoid mode" agent wrapper with Tor, zkAPI and local prompt rewriting — DavideCrapis · 2026-10-16
- Security team demos Signal 1-click exploit technique at Hexacon Paris — moyix · 2026-10-16
- China moves to impose registration and licensing on frontier AI model weights and chip designs — deanwball · 2026-10-16
- Same agent task, two auth models: shared admin login wipes 1M audit rows, per-agent identity gets refused — Prompt Engineering · 2026-10-16
- Binding agent approvals to exact content: a synthetic test for stale permissions — Crescitaly · 2026-10-16