Skip the LLM judge: a deterministic reward function for comparing RL rollouts

ivan_bezdomny · x · 2026-08-21

The author shares an RL evaluation insight: unlike Harvey's LLM-judge approach, he designs a deterministic reward function to compare rollouts, because:

It's more deterministic and much faster;
LLMs aren't inherently good at ranking multiple options into a number — you'd convert binary judgments or checklists back to a score anyway, and N² pairwise comparisons per batch of rollouts are impractical;
The reward function needn't be complete — it just needs to be a good proxy for what your LLM judge would decide, so it can be optimized offline and adjusted after each batch of RL traces.

It doesn't work for all problems, but works well when applicable.

Original post →

Skip the LLM judge: a deterministic reward function for comparing RL rollouts

More from coding & agent