ClawGymII: Training Models Directly via Black-Box Agent Harnesses

智东西 · wechat · 2026-08-18

Core Concept: Decoupling Execution and Optimization

Renmin University of China proposed ClawGymII, a framework for performing black-box reinforcement learning directly through real Agent Harnesses (e.g., ClaudeCode, OpenClaw).

Key Solutions:

Scalable Execution: Uses Sandboxes to isolate task rollouts, supporting massive concurrency. Tools are integrated via MCP without modifying Harness internals.
Trajectory Reconstruction: Addresses fragmented black-box calls with Prefix-Tree Reconstruction, merging shared history and filtering invalid branches (e.g., retries) to form complete training trajectories.
Mix-Harness Training: Enables a single shared policy model to learn simultaneously from heterogeneous Harnesses like OpenClaw and ClaudeCode.

Results:

On Qwen3-30B-A3B, black-box RL via OpenClaw and ClaudeCode improved scores on PinchBench by 17.28 and 11.71 points, respectively.
The framework demonstrated stable improvements across diverse tasks like JobBench and OfficeQA, proving its versatility.

Original post →

ClawGymII: Training Models Directly via Black-Box Agent Harnesses

More from coding & agent