Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation
Mario Sanz-Guerrero, Minh Duc Bui, Manuel Mager, Katharina von der Wense
cs.CL
2026-09-29
Changing only the hidden date in the system prompt moves LLM math accuracy by up to 14 points and swaps the top two on MMLU between consecutive days.
Benchmark scores for LLMs are known to wobble with batch size, numerical precision, and hardware. This paper adds a source nobody had systematically measured: the current date, silently injected into the system prompt by serving frameworks and providers. Users cannot see it, cannot remove it, and it changes every day.
The date looks irrelevant to the questions. The authors verified that MMLU, GPQA, and ARC-Challenge contain no time-dependent items, then reran evaluations with identical configs, changing only the date. GPT-5.1, rerun twice per date for a week, returned identical scores all 42 times; change the date and the score moves.
The task spread is deliberate: multiple-choice is scored from the probability of a single answer token, the smallest surface the date can act on, while longer generation gives the date prefix more chances to bias intermediate tokens, and those perturbations cascade into different final answers.
| Task | Metric | Avg delta | Max delta |
| MCQA (3 datasets) | accuracy | 2.52% | 6.06% |
| GSM8K math | accuracy | 7.75% | 14.42% |
| HumanEval code | pass@1 | 4.81% | 7.32% |
| WMT en-to-cs | BLEU | 1.88 | 2.84 |
Deltas are worst-date minus best-date within 2024; all maxima come from GPT-OSS 120B (MCQA on GPQA).
An explanatory experiment: rewriting the system instruction six different ways produces the same accuracy CV as the date effect (0.78% both). Models are sensitive to any system-prompt change, and the date is one such change, except it happens without the user knowing. A plausible cause is that system prompts stay fixed during SFT and RLHF.
Anyone running evaluations should put this in the protocol: fix the date and report it, or strip it from the chat template. A one-point gap between two models released on different days can be pure noise, and worst-case swings of 7 to 14 points on reasoning tasks are enough to flip conclusions.
Teams building on APIs are more exposed: the injection happens server-side, beyond user control, and GPT-5.1 moved 4 points on GPQA within a single week.
For translation the scale is easy to grasp: WMT progress is usually reported in fractions of a BLEU point, and the date effect reaches 2.84 BLEU.
This is a measurement paper with no new method. What it should change is the evaluation protocol itself, the same way option-order bias worked its way into best practice.
The stated boundaries: coverage is limited in models and datasets; the proprietary evaluation covers GPT-5.1 for one week only; dialogue and summarization are untested; the date was injected in one format, one position, and one year (2024), leaving other formats, positions, and years unexplored; and 365 runs per model per dataset kept the scope tight.
Two caveats. The mechanistic explanation stays at the hypothesis level; the system-instruction experiment is correlational evidence, not proof that fixed system prompts during training are the cause. And 14.42% is a worst case for one model, against a GSM8K average of 7.75%, so the extreme value should not be quoted as typical.