MIRA-AC: Training Autoresearch Agents with Proxy Feedback and Credit Assignment

On October 14, author anirudhg9119 posted a long thread introducing MIRA-AC, a new method for training autoresearch agents. It tackles the delayed credit assignment problem: an experiment's immediate result does not equal the long-term value of the decision behind it. When an agent runs hundreds of experiments, a failed experiment may rule out a hypothesis and pave the way for a later breakthrough, while a quick win may lead to a dead end—so credit cannot be assigned based on immediate results alone.

Confirmed

• The core approach separates ideation from execution, builds the pipeline with a harness, and combines three key ideas: decomposition, value priors, and reinforcement learning.
• MIRA-AC is trained with proxy feedback only and improves real (gold) task results over the untrained MIRA baseline: symbolic regression 50 → 63 problems (recorded elsewhere as 50 → 60-plus), physics discovery 8 → 10, causal discovery 76 instances (material truncated).
• Experiments show the model can learn effectively from its own proxy hill-climbing signals, and proxy-signal training transfers across multiple autoresearch environments.
• Decision-level credit assignment uses only 25% of tokens, keeping compute overhead manageable.

Why it matters

• The work offers a viable paradigm for hierarchical training of "what to research" and "how to do it," showing that agents can learn decision policies with long-term value from proxy signals alone, without costly ground-truth final-result feedback.
• Consistent gains across multiple benchmarks demonstrate the effectiveness of the delayed credit assignment mechanism for long-horizon research tasks, offering design lessons for future autonomous research agents.

2026-10-14 ~ 2026-10-14 · 14 related posts

• Training autoresearch agents with RL by separating what to investigate from how — anirudhg9119 · 2026-10-14
• [source] MIRA-AC: decomposition plus learned value priors fix delayed credit assignment for research agents — anirudhg9119 · 2026-10-14
• Why credit assignment is hard for research agents: failed experiments can enable later breakthroughs — anirudhg9119 · 2026-10-14
• Inside MIRA: a hierarchical research agent separating meta-reasoning from execution — anirudhg9119 · 2026-10-14
• One research decision, one action: redefining the unit of learning in MIRA — anirudhg9119 · 2026-10-14
• Even without RL, separating decisions from execution improves long-horizon inference — anirudhg9119 · 2026-10-14
• A generative critic forecasts remaining research return at each decision boundary — anirudhg9119 · 2026-10-14
• MIRA-AC initializes a shared actor–critic from the pretrained value model — anirudhg9119 · 2026-10-14
• Stitching episodes across context resets: length-adaptive GAE preserves early credit — anirudhg9119 · 2026-10-14
• MIRA-AC results: proxy-signal training transfers across environments using 25% of tokens — anirudhg9119 · 2026-10-14
• MIRA-AC RL details: stitching decisions into one logical episode, updating only the meta-reasoner — anirudhg9119 · 2026-10-14
• [source] MIRA-AC trains research agents on proxy feedback, lifting symbolic regression from 50 to 63 solved — anirudhg9119 · 2026-10-14
• [source] MIRA-AC behavior shift: hypothesis-distinguishing tests up 16.8% to 28%, stalled tweaks halved — anirudhg9119 · 2026-10-14
• Autoresearch agent eval: hypothesis-distinguishing tests up from 16.8% to 28.0% — anirudhg9119 · 2026-10-14