CMU team open-sources Reef, infrastructure for continuously self-improving agents

CMU's Paul Pu Liang and the Human-Agent-Society team (including CMU researchers Ao Qu and Han Zheng) open-sourced Reef, a continual-learning infrastructure whose core idea is that "the inference server itself is a learner" — agents keep improving from their own runtime experience after deployment. Code is available at github.com/Human-Agent-Society/reef.

Confirmed

• Reef exposes standard HTTP endpoints: agents can be downloaded like models, and their model requests are routed to Reef's inference endpoint instead of vendor APIs.
• The key difference: Reef continuously evaluates agent behavior in the background and improves both the harness (orchestration) and model weights it serves, turning inference from the end of a pipeline into part of a learning loop.
• Sharers note the idea echoes Ilya Sutskever's view that future intelligent systems should learn from deployment, which requires redesigning inference serving as learning infrastructure.

Why it matters

Most agents are frozen after deployment. If Reef's "inference-as-learning" approach works, agents could self-improve from runtime signals without manual retraining or redeployment — a notable open-source experiment in continual learning and self-evolving agents.

2026-09-02 ~ 2026-09-03 · 5 related posts

• Reef: Open-Source Infra for Self-Improving Agents via Inference Signals — pliang279 · 2026-09-02
• [source] Reef Open-Sourced: An Inference Server That Continually Learns and Self-Improves Your Agents — pliang279 · 2026-09-02
• Reef: open-source infra that lets agents keep evolving weights and harness after deployment — Scobleizer · 2026-09-03
• Reef open-sourced: infrastructure for agents to continuously self-improve from inference-time signals — pliang279 · 2026-09-03

1 near-duplicate retellings: qinzytech