Resin: Personal Project Lets an AI Agent Write the Forward Pass and Out Pops a Standalone LLM Inference Engine

VoidAsuka · x · 2026-11-17

Developer ekzhang1 unveiled Resin, a runtime for building fast LLM inference engines where all generic parts are reusable — only the model-specific forward pass and kernels need writing per architecture/hardware target.

Workflow: import Resin's core, have an AI agent write the forward pass for your model on GPU/TPU/AMD, and you get a standalone inference engine. The ModelState abstraction (shown in the attached slide) encapsulates model-specific logic with hooks for multimodal embeddings, attention metadata, and CUDA graph capture, supporting cases like diffusion LLMs and encoder-decoder models.

vLLM lead woosukk responded that vLLM is moving the same direction with model-specific code isolation, and Resin 'makes a strong case' for migrating more of vLLM's stack to Rust.

Original post →

More from coding & agent

coding & agent channel →