ProgramAsWeights compiles English function specs into local neural programs
yuntiandeng · reddit · 2026-09-20
Researchers at the University of Waterloo open-sourced ProgramAsWeights (PAW): describe a text function in English, compile it into a reusable neural program, and run it locally—even on CPU—with no external API at inference time. The idea separates task understanding from execution: a large model acts as a compiler that generates weights for a small interpreter.
How it works
- A finetuned Qwen3-4B compiler generates a LoRA adapter for a frozen Qwen3-0.6B interpreter; a neural program = adapter + pseudo-program (cleaned spec + examples)
- Trained on (spec, input, output) triples with gradients flowing through the frozen interpreter; compilation takes seconds
- The same frozen 0.6B base becomes useful for different tasks just by loading different programs
Results
- Built FuzzyBench, a synthetic dataset split by specification
- The 0.6B interpreter hits 73.4% exact-match vs 68.7% for directly prompting Qwen3-32B
- Follow-up work, Compile by Training, uses the generated LoRA as initialization for further accuracy gains
More from Research
- Harvard Professors Admit AI Article Reviews Now Beat the Best Humans — iamfakhrealam · 2026-11-04
- Pre-pretraining gains traced to embedding init: N(0,0.02) alone lifts perplexity 9% — LauraRuis · 2026-11-04
- kalomaze shares loss curve of larger-backbone run on DeepSeek V3.1 base, eval loss down to ~0.82 — kalomaze · 2026-11-04
- Hobbyist rewrites Qwen 0.6B weights with 39.8M tokens, no gradients, in 3h12m — Altruistic_Heat_9531 · 2026-11-04
- SpecPilot agent boosts speculative decoding from 4.46× to 10.87×, no retraining — pmttyji · 2026-11-04
- Autonomous agent designs target-specific MSAs to improve AlphaFold 3 complex prediction — SJTU · 2026-11-04