A CPU Inside an LLM: zero-gradient compilation of exact arithmetic circuits into pretrained weights

Tanmay Sah · hf · 2026-11-20

This paper introduces closed-form, zero-gradient circuit-grafting primitives that compile deterministic computation directly into pretrained LM weights — zero training examples, zero runtime hooks — while bounding per-layer normalization drift and preserving natural-language competence.

Three analytical mechanisms:

Results: an automated 5-stage compiler synthesizes 4–8 digit signed {+,-,×} circuits in constant TC^0 depth. On 6-digit arithmetic, IBM Granite 4.2 3B reaches 99.17% and Qwen 3.5 4B 100.00% ALU-prefix accuracy (100% with an FP32 sidecar), with 98–100% mid-sentence resumption — far outperforming 12.8K-sample LoRA (42.2%/61.7%). A block-sparse sidecar keeps A100 memory at 7.89/10.65 GiB. Boolean and 5-bit Majority/Parity circuits pass at 100%.

Original post →

More from Models

Models channel →