Blind test: deliberative multi-perspective AI system loses to a single frontier model call
Wonderful_Bite1139 · reddit · 2026-10-03
The author built TRAMA 2.0, a Multi-Perspective Deliberative System (MPDS) on top of a frontier model: a single entity that decides how much to think, runs three internal "faculties" over each question, then makes a final decision, with long-term memory, web search, and full logging.
A blind test pitted four systems on the same questions (e.g., how a town should spend €200k once: LED street lighting vs two years of free after-school programs), judged blind by Claude Opus 5.5 against six pre-set criteria:
- The frontier model alone: best reasoning, and it dissolved the dilemma (an ESCo financing the LEDs), in under 60 seconds.
- TRAMA + search: most verifiable answers (sources cited, a vendor flagged, a figure discarded after checking), but with a calculation error and poor readability; 4 minutes.
- TRAMA without search: honest but thin — assumed data was unavailable instead of looking it up.
- A lighter model from another lab: well organized but wouldn't commit to a choice.
Key lessons: search mattered more than deliberation, and three perspectives on the same model failed to catch an arithmetic mistake. Limitations are openly stated (one question, judge from yet another lab), with a 3–5 question follow-up planned.
More from Research
- Kevin Buzzard: AI is solving math humans can't, and mathematicians are grieving — stevenstrogatz · 2026-10-03
- SkillRefiner: Offline skill refinement from your agents' existing execution traces — gregd_nlp · 2026-10-03
- Open pretraining run matches Llama 3.2 1B on ARC-C at ~10% of the cost, author details 4 pitfalls — jon_durbin · 2026-10-03
- MIT's VISTA gives Claude visual memory to clear all 25 ARC-AGI-3 games with 57.4% fewer actions — mark_k · 2026-10-03
- Stephen Wolfram on the future of pure math research in the age of AI — pickover · 2026-10-03
- Neuralink participants have now generated over 50,000 hours of unlabeled neural data for BCI pretraining — XFreeze · 2026-10-03