Resin: Personal Project Lets an AI Agent Write the Forward Pass and Out Pops a Standalone LLM Inference Engine
VoidAsuka · x · 2026-11-17
Developer ekzhang1 unveiled Resin, a runtime for building fast LLM inference engines where all generic parts are reusable — only the model-specific forward pass and kernels need writing per architecture/hardware target.
Workflow: import Resin's core, have an AI agent write the forward pass for your model on GPU/TPU/AMD, and you get a standalone inference engine. The ModelState abstraction (shown in the attached slide) encapsulates model-specific logic with hooks for multimodal embeddings, attention metadata, and CUDA graph capture, supporting cases like diffusion LLMs and encoder-decoder models.
vLLM lead woosukk responded that vLLM is moving the same direction with model-specific code isolation, and Resin 'makes a strong case' for migrating more of vLLM's stack to Rust.
More from coding & agent
- Early hands-on with Gemini 4 Carbon in Antigravity CLI: slow but insightful — iskander · 2026-11-17
- Clinical genomics agent spots two lethal translocations NGS missed, in first peer-reviewed paper — danielmckinn0n · 2026-11-17
- Tsinghua's ActionOPD lifts Qwen3-4B agent success on τ²-bench from 29.2% to 40% — jiqizhixin · 2026-11-17
- Anthropic's daily code merges grew 12x in 4 months, now bottlenecked by human-scale infra — Dr_Singularity · 2026-11-17
- Kanban is a dead end for AI agents, argues dev pushing live reasoning-trace monitoring — kevinkern · 2026-11-17
- Software factory practice: verification infrastructure beats prompt and model choice — vikvang1 · 2026-11-17