A compiler turns Python computation graphs into vanilla Transformer weights with zero training

notforrob · reddit · 2026-07-25

The author built a compiler that turns ordinary Python computation graphs into the weights of a vanilla transformer.

The resulting checkpoint follows a standard Phi-3 architecture and loads in Hugging Face with no custom code or trustremotecode.
The pipeline requires zero training: you describe the computation graph, and the compiler emits transformer weights that execute it.
The write-up frames this as a way to study what algorithms transformers can express, not just what they can learn.
The author notes related work such as RASP and Tracr, but says their goal is different: use ordinary Python and target a stock architecture that works out of the box.
Repo includes twelve runnable examples.

More from coding & agent