A Dario Amodei reply highlights harness-agnostic post-training and attention residuals
peterjliu · x · 2026-07-29
A reply to Dario Amodei praises Anthropic’s post-training work for avoiding overfitting to benchmark harnesses and trying to stay harness-agnostic.
The response highlights Attention Residuals as a notable idea. The claim is that attention’s key advantage over standard RNNs is access to state from previous steps, rather than only the summarized state from the immediately preceding step. The commenter suggests that standard Transformers partially compensate through depth-wise residual connections, and points to this as an innovation not commonly seen in frontier models.
Related event: Anthropic Reportedly Uses Claude for CUDA Coding(2 posts)→
More from Models
- NVIDIA open-weight model ties pplx-decider at #1 on 669-case clinical benchmark — MaziyarPanahi · 2026-10-16
- Claude Code adds idle compaction to save 1h prompt cache on idle sessions — EricBuess · 2026-10-16
- Thrive Holdings launches real-work AI evals from 75+ businesses, benchmarking 13 models — samaysham · 2026-10-16
- Microsoft's MAI-Transcribe now handles 61% of all transcription on OpenRouter — mustafasuleyman · 2026-10-16
- Reflection AI Releases Open-Weight 501B MoE Beam for Agentic Coding on AWS — AWS ML Blog · 2026-10-16
- Dev RL-tunes Qwen 27B against slop with Meta's XAR method, lands top-4 on EQbench — Mysterious_Hearing14 · 2026-10-16