20 text principles lift GPT-4o accuracy 53.2%→67.6% on training-intuition benchmark
ZimingLiu11 · x · 2026-10-24
Posts 4-5 of the Architecture IQ thread report key findings: using Opus to build an evidence-weighted knowledge base, GPT-4o's accuracy rises from 53.2% to 67.6% on 50 held-out questions with up to 20 principles and no weight updates — experience becomes transferable text, though still incremental rather than a science of AI.
The author summarizes four limitations of LLMs' training intuition: it is imperfect or even sub-human in some cases; empirical rather than structured; not maximally condensed; and insensitive to dataset properties.
More from Models
- Abacus AI to launch SuperComputer: text prompt to custom-trained LLM in an hour — bindureddy · 2026-10-24
- One-prompt Minecraft build of an RTX 5090 sparks model-vs-Opus 4.5 poll — adonis_singh · 2026-10-24
- Abliterated DeepSeek running on a Pi is the AI power-user flex of the day — EXM7777 · 2026-10-24
- Tesla engineer: Cybercab camera tuning done with Grok, coding fixed-function hardware directly — yunta_tsai · 2026-10-24
- VulnPR-100: DeepSeek V4.1 Flash finds 54 vulns for $7.53, crushing rivals on cost — teortaxesTex · 2026-10-24
- Kimi K3.1 Pro costs $1.03 median per task but lands just off Agent Arena's Pareto frontier — arena · 2026-10-24