Developer builds local agent "Secret Agent Bubbles": Qwen works by day, self-trains by RL at night
cephaloform · x · 2026-08-26
cephaloform shares the full design of their first local agent, deliberately built without looking at others' implementations.
- Daytime: a Qwen-based agent takes tasks over Discord and calls tools to complete them.
- Nighttime: Qwen becomes a reward model, extending 10-message windows and reporting sub-tasks, lengths, and reward estimates.
- Value model: a separate agent predicts reward and turn count from embedding-based memory weighted by date/relevance; the prediction deltas form the policy-gradient advantage, and the value model's own predictions are trained with an accuracy reward.
- Mutual understanding: 30% of rollouts get an extra check of whether user and agent understood each other, averaged into the advantages.
- Training uses OAPL (batch 64) plus a small imitation loss for an implicit world-model, as in ECHO.
Related event: Developer Builds Local Agent Bubbles: Works by Day, Self-Improves by Night(2 posts)→
More from coding & agent
- AI workflows fail over time; Agents work better as managers than workers — Even-Explanation-133 · 2026-08-26
- ComfyUI workflow automates Minimax H3 video generation and timing benchmarks — GeroldMeisinger · 2026-08-26
- Granite 3B outperforms larger models in agent tasks: user test — D3VAUX · 2026-08-26
- OpenClaw enables multiple concurrent AI agents with independent cursors — soumitrashukla9 · 2026-08-26
- AI Short-Drama Skills: Novel to Production Pipeline — aigclink · 2026-08-26
- Resuming sessions over-counts agent costs by factor of N — donk8r · 2026-08-26