HF live-streams agent RL: LoRA adapters trained inside real harnesses like opencode
yb2698 · x · 2026-11-06
Hugging Face open-sources and live-streams an agent RL protocol: open models (LFM2.5-24B-A2B, LFM3-1.7B, TwiL-8B) run unmodified inside real agent harnesses (pi, opencode, hermes) while a small LoRA adapter (rank 32, all linear layers) is trained per model with TRL — testing whether RL on verifiable tasks inside a harness improves the model under that same harness.
Early signal: a 20-step LFM3-1.7B smoke run on 150 held-out tasks showed +9.4 ± 6.3 points pass@1 on opencode (69%→79%), +6.7 on hermes, flat on pi; TwiL-8B, never trained on agentic tasks, also learns and gets more efficient. A safeagents proxy blocks undeclared tools, unknown hosts and high-severity calls, scoring blocked rollouts 0. Envs, rules, recipe, per-checkpoint evals and all blocked calls are open; built with TRL, OpenEnv, Harbor and vLLM, with three trainers per model sharing one vLLM.
More from coding & agent
- Dataset Builder agent: one prompt to keep ML training datasets fresh monthly — nlarusstone · 2026-11-06
- Compound Engineering: automation makes work less synchronous, not less human — every · 2026-11-06
- Chaining Claude connectors to automate expense filing end to end — dfeinition · 2026-11-06
- From idea to iPhone app in under 15 hours, with an agent driving the browser — jacalulu · 2026-11-06
- Codex Desktop App Slammed for Ridiculous Lag on Tab Switch and Typing — var_epsilon · 2026-11-06
- China's major tech firms sign eight-point voluntary convention governing AI agents — pstAsiatech · 2026-11-06