New arXiv paper extracts the cruxes of LLM forecasts by contrasting 30 predictions
universeinanegg · x · 2026-10-22
arXiv:2610.21006 tackles a validation problem: when an LLM writes a ten-page rationale that top-model API prices halve by 2027, which arguments actually drive the conclusion? Cruxes can only be identified by contrasting forecasts.
The method: generate 30 forecasts for the same question (e.g., 7–15% yes from Claude Opus), extract pairwise differences, ignore paraphrases, and aggregate genuine cruxes — such as when compute shortage eases, whether cheaper near-equal rivals weaken premium pricing power, and whether labs choose margins over market share (85+ cruxes in the example). Key insight: cruxes are properties of the distribution of forecasts, not of any single forecast.
More from Research
- MIT launches course on self-evolving AI spanning context, harness, model and task evolution — pliang279 · 2026-10-23
- Daily Experiment Planner agent: one prompt to save scientists hours of lab prep — nlarusstone · 2026-10-23
- Phylo partners with Bayer to bring Biomni Lab AI co-scientist platform to its research teams — KexinHuang5 · 2026-10-23
- Study: 96% of ChatGPT memories are created unilaterally by the system — alex_verem · 2026-10-22
- AI2's OlmoEarth embeddings beat AlphaEarth and Tessera v2 on 10 of 11 evals — allen_ai · 2026-10-22
- Testing decision models like Jev and Drex Max for research triage: Drex Max heads to production — omarsar0 · 2026-10-22