A 44-model study finds JSON prompts can make LLM answers more standardized
vista8 · x · 2026-07-27
A discussion thread summarizes a paper testing 44 models and finds that output format changes model diversity in a measurable way.
- When asked to “say a word,” the most common answer across the 44 models was serendipity.
- In normal dialogue, responses were diverse about 41% of the time.
- When the same request was forced into JSON, diversity rose to 64%.
- The argument is that JSON/XML-style prompts push models toward standardized answers, because they have been heavily trained on structured output in post-training.
- The takeaway: if you want richer variety, avoid forcing overly rigid structured formats unless you actually need them.
Related event: Structured Outputs Reduce LLM Diversity, Study Finds(2 posts)→
More from Research
- Anticipating the K3 Model Report: Data Recipes and Large-Scale Ablations — stochasticchasm · 2026-07-28
- Kimi K3 claims 2.5× better scaling efficiency with a three-axis architecture — suchenzang · 2026-07-28
- Attention Residuals may expose cross-layer information flow directly for interpretability — tokenbender · 2026-07-28
- PRO-LONG hits 97.4% on public ARC-AGI-3 for $1,750 with a log.txt memory harness — GregKamradt · 2026-07-28
- ReactBench v1 ranks GPT-5.6 at about 53% and Opus 5 at 49% on realistic React tasks — aidenybai · 2026-07-28
- fchollet on iLands: Open-Ended Survival Environments Are Key to Shaping AI Intelligence — fchollet · 2026-07-27