POSTECH paper: weight-space programming doesn't need Transformers — 680x smaller, 17x faster
CatAstro_Piyush · x · 2026-11-17
POSTECH researchers Hyunjong Ok and Jaeho Lee argue Transformers are the wrong default for weight-space programming. When an LLM writes network weights directly (no gradient training), an associative-memory architecture of 0/1 fact tables + rules — one fact per row — matches ARC accuracy with 680–1800x fewer parameters than the best Transformer the same LLM wrote, and runs up to 17x faster on GPU with a table-tuned memory design. Paper and code are public.
More from Research
- MIT's Jacob Andreas trains models to explain their own computations — MacrinePhD · 2026-11-17
- ICLR reviewer flags AI-written reviews: a third of his batch undisclosed — vaiter · 2026-11-17
- IFM releases K2-Horizon: six fully open LLMs up to 375B MoE, trained on ~10T synthetic tokens — IFM · 2026-11-17
- PKU & Tencent Hunyuan's MovieRoll extends streaming video generation to 60s with 78% KV-cache savings — Yunyang Ge · 2026-11-17
- UMM-JEPA: predicting continuous perceptual representations beats discrete codes for unified multimodal models — Yucheng Zhou · 2026-11-17
- ScaleVLA scales VLA models to 552B parameters: 10 demos beat 2B backbone trained on 50 — GalaxyGeneralRobotics · 2026-11-17