Pi-100 steps hit best tradeoff: LFM2.5 LoRA cuts 37% tool calls
yb2698 · x · 2026-10-30
Author compares LFM2.5-24B-A2B LoRA checkpoints against the base model on the same tasks and eval harness:
- pass@1: the Pi run (lr 1e-5) lands at -8 points vs baseline at step 110, while the claude-code run hits +4 at step 100; anything within ±5 points is noise at 100 tasks.
- Tool-call efficiency: Pi-100 steps is the best tradeoff — about 37% fewer tool calls on tasks both solve, with accuracy largely retained.
- The claude-code version showed no meaningful boost yet; iteration continues.
Training used LoRA + RL on a single B300 GPU in 24h, splitting VRAM between a vLLM server and the training process.
Related event: LFM2.5-24B gets LoRA adapters for Pi and Claude Code(4 posts)→
More from coding & agent
- Python gotcha breaks LLM evals: "İSTANBUL".lower() doesn't return "istanbul" — irombie · 2026-10-30
- JetBrains' open-source AI IDE ThinkRail hits 6.3k GitHub stars on launch — abhishek__AI · 2026-10-30
- JetBrains open-sources ThinkRail, an AI IDE where multiple coding agents work in parallel and specs persist — abhishek__AI · 2026-10-30
- "Your LLM evaluation is lying to you": temp-0 Qwen3-235B gives 80 different answers in 1000 runs — irombie · 2026-10-30
- A 4B Model Verifies a Coding Agent 70x Its Size at Just 4% of Steps, 9-13% Extra Cost — petrusenko_max · 2026-10-30
- Static RAG eval sets go stale: treat your eval dataset as a pipeline, not a file — paulabartabajo_ · 2026-10-30