One BOS token changed 6 of 15 LLM responses in a controlled experiment
Full-Travel857 · reddit · 2026-11-08
A Reddit user ran a controlled experiment on Bielik-Minitron 7B (Q4KM, Ollama, temperature 0, seed 42), discovering that Ollama's chat template for this model omits the BOS token <s> present in the GGUF template.
Design: 15 tasks (Polish, JSON, Python) × three variants — Ollama template, raw prompt without BOS, raw + <s> — each repeated 3 times (135 trials), with the model unloaded and zero prompt-cache reuse before every trial, isolating BOS as the only variable.
Results:
- Template and raw-without-BOS produced identical outputs across all 15 tasks;
- Adding <s> changed 6 of 15 responses (40%), with prompt length up exactly one token;
- Only one quality verdict flipped: Polish diacritics went 0/3 → 3/3, overall 11/15 → 12/15; other changes were punctuation, explanation length, JSON formatting, or loop style.
The author explicitly doesn't claim BOS universally improves quality — only that a single input token sufficed to change 40% of responses, a key caveat for building reproducible model test setups.
More from Research
- Agent S4 lets agents evolve reusable computer-use interfaces from their own task experience — burkov · 2026-11-08
- Stanford Prof Anshul Kundaje on Weight Space Editing: Replicated Across Models, Limits Still Unclear — anshulkundaje · 2026-11-08
- CrowdGPT Wants to Train an LLM Collaboratively—No Datacenter, Fully Open Data and Weights — Vxtzq1 · 2026-11-08
- FDT from the paper defects in almost-twin prisoner's dilemma, shown via ChatGPT — xuanalogue · 2026-11-08
- Open book on historical newspaper data ships with R workflows and AI tutor — burkov · 2026-11-08
- Lamport Sieve audits robot teleop data: deleting bad demos made training worse — R4nsack · 2026-11-08