Evolving with Open Worlds: Towards Recursive Self-Improvement for Future Forecasting
Zhipeng Chen, Yunsen Liang, Yanxi Chen, Hongkang Zhao, Lingyun Feng, Ruiyang Ren, Guoxin Chen, Jia Deng, Feng Chang, Chuan Hao, Ran Tao, Jian Yang, Ji-Rong Wen, Bryan Dai, Wayne Xin Zhao
cs.CL
2026-10-21
EvoForesight separates search from prediction and recycles outcomes into weights, skills, and tasks. It led FutureX for three weeks, rating up to +195% and BT +10.7.
Recursive self-improvement needs fresh tasks and an automatic way to tell whether the system got better. A fixed benchmark supplies neither. The items never change, and the answers can leak into training.
Forecasting events that have not happened yet supplies the missing loop. Questions come from the world as it unfolds, the label arrives when the event resolves, and the distribution moves when the world does. Most forecasting work optimizes a single shot. Self-training has no external ground truth. Continual learning usually sees a pre-cut task list. Reinforcement learning on reasoning optimizes the reward in front of the model. EvoForesight, from Renmin University’s Gaoling School and IQuest Research, turns forecasting into a loop that updates weights, skills, and the question generator from the same trajectories.
Prometheus builds the task stream. Each day it pulls news and schedules from 125 sites, writes questions with templates and with Codex open discovery, then filters for time window, duplicates, and source diversity. The in-house eval asks only about events that resolve the next day. Codex retrieves the public outcome and scores the forecast, with adversarial question writing switched off so the comparison stays fixed. FutureX is the external weekly board and ranks systems with an official sliding-window Bradley-Terry model against outcomes that actually occur.
Branchus separates gathering evidence from making the call. A search team (scheduler, searcher, critic) browses only while the question is still open, so what it finds depends on when it runs. A predict team reads the cached trajectories and outputs a probability. Skills have four levels, from a cross-domain rule down to a concrete step. Hand-written seeds start the library. Later entries are distilled from trajectories.
Athena spends each trajectory three ways. On weights, GRPO trains only the predict team. The reward is whether the top-probability outcome matches the resolution, and the advantage is normalized inside one question’s sample group. Successful trajectories are also used for fine-tuning (RFT). On skills, the weights stay frozen. A judge scores the search roles without seeing the outcome, then filters those notes into skills. On the environment, error rates by question type become task-writing skills, so the next round pushes the categories the agent just missed. Questions still have to be objectively resolvable.
After resolution the answer is on the web. If the predict team could search again, reinforcement learning would learn to look the answer up. Caching freezes the pre-resolution evidence for every later update.
On FutureX, the self-improved forecaster placed first among all systems for three consecutive weeks, with rating gains up to 195%. The results text never gives the start and end of that 195%. Across four weekly windows, Branchus sits at or next to the top.
The in-house board has 23 harness-model pairs. Inside each of the four base-model blocks, Branchus has the highest Bradley-Terry score. The margin over the best baseline in the block runs from 10.7 down to 0.5.
| Base model | Branchus BT | Best baseline in block | Overall rank |
| Qwen3.8-Max | 62.2 | MAS 51.5 | 1 |
| GPT-5.6-Sol | 57.9 | MAS 55.0 | 2 |
| DeepSeek-V4-Pro | 56.4 | MAS 55.9 | 3 |
| Claude-Opus-4.6 | 53.1 | OpenClaw 51.2 | 9 |
Choice accuracy does not separate them. On DeepSeek-V4-Pro, MAS scores 72.7% and Branchus 71.9%. On Claude-Opus-4.6, OpenClaw scores 73.8% and Branchus 73.0%. Calibration is where the gap shows. On Qwen3.8-Max, Branchus has the best Brier score in the block, 0.336, and the best relative WIS, 0.737. On Claude its Brier of 0.331 is also the lowest in the block. On GPT-5.6-Sol its relative WIS of 0.865 is the best. On DeepSeek-V4-Pro, MAS is slightly sharper on Brier (0.340 vs 0.341) and relative WIS (0.823 vs 0.835), yet Branchus still leads on Bradley-Terry.
The parameter ablations use smaller models. RLVR on Qwen3-4B-Thinking and Qwen3-8B raises choice accuracy by 1.3 points on each, and raises the numeric score 1-bWIS by 0.31 and 0.38. Two rounds of RFT on GLM-4.7-Flash, as the pool grows from 600 to 1,000 items, move Brier from 0.651 to 0.636 to 0.630 and relative WIS from 1.02 to 0.99 to 0.98.
Skills leave the weights alone. Three nested pools, 88, then 186, then 262 trajectories, take Brier from 0.135 to 0.122 and MAPE from 2.80% to 2.28%. Coverage grows from 168 domains and 173 tasks to 479 domains and 505 tasks. On templated questions, tool calls fall from 32 to 18 per question, with MAPE at least as good as the skill-free baseline. On open questions, calls rise from 30 to 34 and MAPE falls from 3.40% to 2.31%. Distilling one trajectory costs about 303,000 tokens.
Weakness-targeted questions make the same checkpoint worse in the way the design wants: selection accuracy 0.60 to 0.53, Brier 0.52 to 0.58, AUROC still about 0.68 (0.678 to 0.68). Those same 262 trajectories yield 1,403 skills against 1,312 without the targeting. RFT on the harder items, compared with plain RFT, lowers Brier by 1.6%, raises AUROC by 2.7%, tightens relative WIS by 1.0%, and improves Md-sAPE by 4.0%.
The reusable piece is the split. When a task expires and the answer later becomes public, lock retrieval to the open window and train only on the frozen packet. Otherwise reinforcement learning collapses into looking up the answer.
Skills are the cheap lever. A few dozen to a few hundred live trajectories already change search: repeated question shapes need fewer tool calls, unfamiliar ones get a wider search. A skill can only reorganize behavior some trajectory already showed. Weight updates cost more and change how the model weighs evidence it already has. The paper’s compute recipe is to distill every round and update the predict team only in occasional bursts.
Accuracy often moves by about a point. Bradley-Terry and calibration move more. Two or three rounds is a short recursion. This is a forecasting-system gain, and it is still far from a general self-improvement result.
The authors already bound the loop. Where resolvable, informative events are scarce, or depend on a local culture, trajectories stay thin. The calendar sets the pace, so questions that take months make each round slow. Skills from one regime can misfire after the world drifts, and the system does not yet forget, time-weight, or detect failed skills. The environment only sees categories the agent already got wrong. Overconfidence that never shows up as a wrong pick never becomes the next batch. Parameter updates also do not yet stack round after round the way the skill library does.
A few headline claims sit loosely on the tables. The 195% rating gain is not given as a start score and an end score in the results section. The paper says the four Branchus setups occupy the overall top three, but Branchus on Claude-Opus-4.6 is 9th in Table 1. The +1.3 accuracy from RLVR was measured on 4B and 8B models, while the leaderboard uses much larger models, and the two experiments are not in one table. Brier scores near 0.13 in the skill plots and near 0.33 on the main board are different slices. In-house labels depend on Codex retrieving the outcome, so a bad page dirties the reward. The training reward is binary hit-or-miss, while the advertised gains are mostly calibration, and that mismatch is not tested on its own. Leakage was not measured. The contamination defense is the cache design.