2026-07-29
OpenReviewer fine-tunes Llama-3.1-8B on 79K real reviews to write ML paper reviews; scores match humans (5.4 vs GPT-4o 7.7), 55.5% exact match on 400 papers.
Large models can already write paper reviews, but they do it too gently. Prior work keeps finding that general-purpose models like GPT-4 lean positive: their recommendation scores run systematically high, and they treat reviewing as flattering the author. For an author who wants honest feedback before submission, that uniformly rosy review is close to useless, and it can mislead them about where their manuscript actually stands.
OpenReviewer targets exactly this bias. It trains a model whose criticism level and score distribution both track real human reviewers. The framing is a self-service pre-submission feedback tool, explicitly not a replacement for human peer review.
The core is Llama-OpenReviewer-8B, a full fine-tune of Llama-3.1-8B-Instruct. Training data comes from OpenReview: about 36,000 papers and 141,000 reviews from ICLR and NeurIPS (2022 onward), keeping only high-confidence reviews, which leaves about 79,000. Papers are converted from PDF to markdown with Marker (handling equations and tables), keeping the main text and references and dropping the appendix.
The fine-tuning details are concrete: 3 epochs, effective batch size 64, learning rate 2e-5, sequence length pushed to 128k to fit a whole paper, bf16 precision, about 34 hours on 64 A100 80GB GPUs. The system prompt borrows from the ICLR 2024 reviewer guide and fixes markdown output with a configurable section template; the user prompt is minimal, just "Review the following paper:" plus the full text. The whole system is a Gradio demo on HuggingFace Spaces: upload a PDF, it auto-converts to markdown, click once, and the review streams out.
On a 400-paper test set from NeurIPS 2024 and ICLR 2025 (the two most recent venues, to avoid leakage), it is compared against four general-purpose models. Recommendations are normalized to 1 (strong reject) through 10 (strong accept):
| Model | Exact match (hits at least one human) | Avg. recommendation |
| Llama-3.1-8B | 14.0% | 8.1 |
| Claude-3.5-Sonnet | 15.5% | 7.6 |
| GPT-4o | 23.8% | 7.7 |
| OpenReviewer | 55.5% | 5.4 |
Human reviewers also average 5.4, identical to OpenReviewer; the general models sit between 6.9 and 8.1, with Llama-3.1-8B as high as 8.1, which would mean accept for most papers. On average absolute recommendation error, OpenReviewer is 0.96 versus 2.34 for GPT-4o. In a preference arena judged by GPT-4o, OpenReviewer's win rate ranges from 60% (against GPT-4o) to 76% (against Llama-3.1-70B).
An 8B model beats general models several times its size on this task, and the lever is specialized fine-tuning, not scale.
The direct takeaway: on a task with clear domain norms like reviewing, specialized fine-tuning beats simply scaling up. For individual authors it is a free tool for fast, structured criticism before submission, with both model and demo open-sourced. For people building LLM applications it is a clean case of using domain data to correct a general model's bias, showing that a small model with the right data can overtake a large one on a narrow task.
The applicability is narrow: training data comes from only two machine-learning venues, and performance will drop on other fields or review templates.
The authors list a string of limits. On data, only ICLR and NeurIPS from 2022 onward, a single domain, and some non-anonymized versions were used, which risks information leakage. On the technical side, quality depends on the PDF-to-markdown step (Marker), figures and images are a weak point, and the model is only 8B, with weaker document understanding than larger models. On evaluation, the test set is just 400 papers (limited by commercial API costs), metrics are mostly automatic with no detailed human analysis, and only a few baselines were chosen.
The premise of the evaluation also deserves scrutiny. The paper assumes that similarity to human reviews equals quality, but human reviews themselves have limited quality control and carry their own biases; using GPT-4o as the judge for "who resembles the human reviews more" lets a model already known to lean positive decide the contest, and the authors do not address how trustworthy that judge is. OpenReviewer's recommendation matching the human average exactly, both at 5.4, is suspiciously clean, and may just mean it learned to regress toward the training-set mean rather than judge each paper.