A 44-model study finds JSON prompts can make LLM answers more standardized
vista8 · x · 2026-07-27
A discussion thread summarizes a paper testing 44 models and finds that output format changes model diversity in a measurable way.
- When asked to “say a word,” the most common answer across the 44 models was serendipity.
- In normal dialogue, responses were diverse about 41% of the time.
- When the same request was forced into JSON, diversity rose to 64%.
- The argument is that JSON/XML-style prompts push models toward standardized answers, because they have been heavily trained on structured output in post-training.
- The takeaway: if you want richer variety, avoid forcing overly rigid structured formats unless you actually need them.
Related event: Structured Outputs Reduce LLM Diversity, Study Finds(2 posts)→
More from Research
- Yale PhD student open-sources his paper figure scripts, packaged as a Skill for Claude Code and Cursor — burny_tech · 2026-09-23
- AI models now match superforecasters on ForecastBench; rematch set for October — burny_tech · 2026-09-23
- Dev uses Opus 5.5 with Lean to formally verify Claude Agent SDK, yielding 16 bug-fix PRs — bcherny · 2026-09-23
- Mathematicians, not just LLMs, made AI's math breakthroughs possible, scholars argue — tak3sh8 · 2026-09-23
- AI-enabled drug discovery cuts discovery time by 15-80%, McKinsey research finds — menhguin · 2026-09-23
- Gemini training details dissected: groupwise reward redistribution to fight reward hacking — nrehiew_ · 2026-09-23