ARC-AGI-3 maintainers explain provider adapter harnesses: vendor-recommended settings per model
mhmazur · x · 2026-11-18
Responding to "what does provider adapter concretely mean?", an ARC-AGI-3 maintainer laid out the details:
- Standard harness: models can carry forward notes between turns, and long conversations are truncated
- Provider adapter harnesses: use each provider's recommended settings. For OpenAI that means preserving opaque reasoning between turns and auto-compacting long conversations; separate adapters exist for Anthropic, Gemini, open-source models, and others
- All provider adapter harnesses are implemented in the open-source ARC-AGI-3 benchmarking repo
- The design was introduced in an earlier blog post about benchmarking GPT-6 Astra
This clarifies why the same model can score differently under different harnesses — key context for reading ARC-AGI-3 results.
More from Models
- Marin releases 535B-A23B pretraining checkpoints, admits benchmark contamination slipped past decontamination — dlwh · 2026-11-18
- Marin releases all 57 intermediate checkpoints of 535B-A23B MoE trained on 18.2T tokens — dlwh · 2026-11-18
- Microsoft's Decision-1.1 decision model lands on Vercel AI Gateway with structured outputs — marlene_zw · 2026-11-18
- GPT-6.2 Sol voxel benchmark: mostly a regression vs 6.1, but much faster and cheaper — Angaisb_ · 2026-11-18
- Karpathy: Google's undistilled Gemini Carbon thinks differently, and it's the only full-stack AI company — beffjezos · 2026-11-18
- SpaceX CFO hails Grok Bot as 'probably the best agentic AI solution out there' — XFreeze · 2026-11-18