Google-led T2J-Bench and TwinHarness lift PyTorch-to-JAX migration reward 11.4% across 63 repo tasks
jiqizhixin · x · 2026-11-16
Researchers from Google Cloud AI Research, UIUC, USC, Notre Dame and Google DeepMind introduce T2J-Bench and TwinHarness (arXiv paper) to verify PyTorch-to-JAX migrations.
The problem: a rewritten model can run successfully while silently changing gradients, optimization trajectories, or inference-cache behavior — and a JAX interface can even hide leftover PyTorch dependencies.
How TwinHarness works:
- A Diagnose–Migrate–Verify loop: localize discrepancies, edit candidate JAX/Flex code and checkpoint conversion, then verify against the runnable PyTorch source
- Code state and validated diagnostic evidence are maintained separately; the agent only sees task-visible resources
- Evaluation runs offline without PyTorch, and probe failures are distinguished from candidate failures
Results: with Claude Code and opus-5 held fixed, mean migration reward rises from 0.649 to 0.723 across 63 repository tasks and three seeds — an 11.4% relative improvement.
More from coding & agent
- Nous community ships mnemos: a self-curated, fading memory plugin for Hermes agents — RileyRalmuto · 2026-11-16
- Antigravity doubles context window for Gemini 4 models — MickeySteamboat · 2026-11-16
- Indie dev cuts $400-600/month by ditching Vercel, Ahrefs and Sanity — yihui_indie · 2026-11-16
- Swarms' AgentRearrange orchestrates multi-agent pipelines via a single flow string — KyeGomezB · 2026-11-16
- Mapping 11 weeks of coding agent sessions (8,000+ messages) as a river — dbreunig · 2026-11-16
- Microsoft open-sources TokenEconomizer, cutting coding agent costs 18–31% — aigclink · 2026-11-16