Livestreamed RL training of LoRA harness adapters for Hermes, pi and opencode, fully open-sourced

yb2698 · x · 2026-11-06

The author is livestreaming RL training of LoRA adapters (rank 32, all linear layers) for three coding agent harnesses — Hermes, pi and opencode — on LiquidAI LFM2.5-24B-A2B (MoE, 2B active params), LFM3-1.7B, and experimental TwIL-8B.
Core question: does RL on verifiable tasks inside a given harness make the model better under that same harness? Prior iterations showed encouraging signals on LFM2.5-24B-A2B.
The live dashboard tracks reward curves, scheduled held-out pass@1 evals per checkpoint, and tool-call costs; each model's three trainers share one vLLM.
Every tool call passes through a safeagents proxy: undeclared tools, unknown hosts, or high-severity rule hits get blocked and the rollout scores 0; lower-severity matches are logged as suspicious.
RL environments, reward hacking attempts, eval results, and the training protocol will all be open-sourced.

More from coding & agent