Animation Bench: GPT-6.1 Sol tops coding agents on web animation reconstruction at $0.46/task
himanshustwts · x · 2026-10-02
Physera released Animation Bench v1.0, testing frontier coding agents on reconstructing behavior (not just looks) of 48 real web animations from 32 commercial sites, 240 reconstructions total. Key results:
- Leaderboard: GPT-6.1 Sol (0.606, Motion 0.512), GPT-6 Astra (0.594), Claude Fable 5.1 (0.548), GPT-6 Sol (0.516), Claude Opus 5.5 (0.507)
- GPT-6.1 Sol won 14 of 48 tasks at $0.46/task, far cheaper than GPT-6 Astra's $3.04; GPT-6 Sol costs $0.45
- Caveat: Sol 6.1 was added Oct 1 under the same tasks/agent/scorer, but recording conditions alone can shift the 48-task mean by 0.01, so its lead over Astra is within uncertainty
- Scores are 0-1 reproduction quality, not success rates; the site offers frame-locked reference vs. model overlays
Related event: GPT-6.1 Sol Tops Animation Bench for Web Animation Reconstruction(2 posts)→
More from coding & agent
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16
- AI agent deletes 43,599 lines of OpenClaw test code in one PR — vincent_koc · 2026-11-16
- JPMorgan shows Jarvis, its internal agent assembly line, at Interrupt NYC — LangChain · 2026-11-16
- Dev clarifies: Apple doesn't ban feature flags, only fooling App Review — talkaboutdesign · 2026-11-16
- Antigravity Matches Claude Cowork in Tests, While Gemini Tasks' Purpose Remains Unclear — sebajun2 · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16