WeightBench: New Benchmark Tests LLMs as Weight-Space Programmers
Aayush Marasini · hf · 2026-11-19
A new Hugging Face benchmark, WeightBench, evaluates LLMs on Weight-Space Programming: reading, modifying, and generating neural network weights. The eval harness puts the researcher agent in a restricted CPU sandbox, accesses the subject model via a budgeted-probe Weight API, and uses append-only hash-chained metrics with auditing. Charts show clear zero-shot capability scaling across 42 tasks: Claude 5.5 Haiku/Sonnet/Opus score 0.402/0.466/0.535 Macro SWB.
More from Research
- New essay makes the case for traditional social science in studying AI agent interactions — RobbWiller · 2026-11-19
- If a model generates valid math blackboards, does it understand math? — francoisfleuret · 2026-11-19
- The Zvi on GTA 6's 99 Metacritic score: 'Welcome to 2026. We need better benchmarks.' — NathanpmYoung · 2026-11-19
- Stanford blood test ages 11 organs from 5,000 proteins; Helios launches 1,000+ biomarker home test — jgooten · 2026-11-19
- Kaggle lays out roadmap to fix AI benchmarks: relevance, saturation, verification, infrastructure — MeganRisdal · 2026-11-19
- Researcher: my real peer review is asking the world's top 2-3 experts, conference reviews are noise — taiyasaki · 2026-11-19