Reinforcing Agentic Creativity in Scientific Ideation with Night Science
Priyanka Kargupta, Silviu Cucerzan, Shweti Mahajan, Allen Herring, Jiawei Han, Ryen W. White, Sujay Kumar Jauhar
cs.AI, cs.CL
2026-09-29
Trains Qwen3 agents with GRPO to learn creativity at action, process, and outcome levels; originality gains up to 66 points, unreproducible by raising temperature.
LLMs improve fastest on tasks with verification signals, code and math reasoning in particular. Open-ended scientific ideation is the opposite case: low-entropy bias pushes models toward high-probability patterns from training data, so generated ideas come out homogeneous and predictable, with nontrivial plagiarism rates. Existing research agents model discovery as a tidy pipeline (retrieve, hypothesize, test, refine), which maps to "day science". The other half, "night science" (loosely structured association, chance encounters across fields, acting on intuitions that challenge default assumptions), is mostly absent: search, debate, and write are treated as fixed behaviors, and creativity is only an attribute of the final idea. Nothing controls how creatively an action is performed or when to deviate from convention.
The paper's position: creativity is not sampling stochasticity. It can be represented in levels and learned with RL.
AI Night-Scientist is a ReAct-style agent. Each step selects an action plus a creativity level, executes it, and appends the result to context; trajectories run up to 5 steps. Creativity lives at three levels:
Training uses GRPO (verl) on Qwen3-8B/14B-Base. The reward lands only on the final proposal: it is decomposed into atomic ideas scored for precedence (distance from prior work), feasibility (credibility of the plan, minimum across phases), and relevance (fit to the problem), averaged with equal weights, with GPT-4.1 as judge and retrieved literature as grounding. Two optional add-ons: stochastically swapping the chosen action early in training with decaying probability, mimicking serendipitous encounters; and a process reward scoring intermediate actions for exploration and contribution.
The load-bearing choice is writing creativity levels in natural language instead of as a numeric knob. Temperature only adds noise to the distribution; "search distant domains" specifies semantically which direction to deviate, which is what lets RL learn which deviations pay off.
The dataset is NSF CSE awards from 2018 onward, 4,414 train and 491 test, and the model sees only the award title. Evaluation covers three things: pairwise win rate against GPT-5.1-reconstructed reference proposals judged by SciJudge-30B (a citation-impact predictor), originality judged by GPT-5.1 against the closest retrieved papers, and coverage over research paradigms and contribution types.
| Method | Citation win rate | Originality win rate |
| Qwen3-8B zero-shot | 0.46% | 2.53% |
| GPT-4.1 + creativity prompt | 3.00% | 14.02% |
| Qwen3-8B + temp mapping + RL | 11.52% | 19.82% |
| Qwen3-8B + ReAct + RL | 24.94% | 46.60% |
| Night-Scientist-8B | 29.89% | 56.32% |
| Night-Scientist-14B | 33.18% | 68.68% |
Against zero-shot Qwen3-14B, the 14B model gains 32.03 points in citation and 66.15 in originality; scaling the zero-shot model from 8B to 14B barely moves anything, so capacity only pays off once the creative policy is learned. Under the identical reward, the temperature variant trails by 36.5 originality points. ReAct+RL recovers much of the gap, and semantic levels add another 9.72. Dropping debate and spark (search + write only) costs 22.99 originality points, so retrieval augmentation alone does not explain the gains. In direct head-to-head comparisons, Night-8B beats GPT-4.1 on 74.95% of citation and 86.96% of originality judgments. Research-paradigm coverage rises from 0.738 to 0.943, about 1.4 extra effective categories out of seven, with positive gains across all 36 evaluated domains. Process rewards trade the other way: originality up another 5.29 points, citation down 3.69, coverage narrower, so the primary setting stays outcome-only.
This turns "LLMs lack originality" from a lament into a trainable problem, with an ablation chain that cleanly separates the contributions of temperature, multi-step policy, retrieval, and semantic creativity levels. The mechanism, describing the direction and degree of behavioral deviation in natural language and letting RL learn when to apply it, is task-agnostic; any long-horizon agent balancing exploration and exploitation could adopt it. Code is at microsoft/ainightscientist. Honest framing: these are results on 8B/14B bases, 5-step trajectories, and 491 test awards, a mechanism contribution rather than a system that replaces a PI.