WeightBench: New Benchmark Tests LLMs as Weight-Space Programmers

Aayush Marasini · hf · 2026-11-19

A new Hugging Face benchmark, WeightBench, evaluates LLMs on Weight-Space Programming: reading, modifying, and generating neural network weights. The eval harness puts the researcher agent in a restricted CPU sandbox, accesses the subject model via a budgeted-probe Weight API, and uses append-only hash-chained metrics with auditing. Charts show clear zero-shot capability scaling across 42 tasks: Claude 5.5 Haiku/Sonnet/Opus score 0.402/0.466/0.535 Macro SWB.

Original post →

More from Research

Research channel →