NVIDIA open-sources TensorRT Model Connect, two commands to TensorRT inference
JFPuget · x · 2026-08-19
NVIDIA open-sourced TensorRT Model Connect (Public Preview): supported HuggingFace models can reach end-to-end TensorRT inference in two commands, no intermediate ONNX export, with the bundle runnable via native C++ APIs. The entire project—implementations, perf tuning, tests, integrations, docs—was built with OpenAI Codex agents, with humans directing and reviewing.
TVM creator Tianqi Chen highlighted first-class TVM-FFI support, letting custom DSL and agent-generated kernels run inside TensorRT. The official tutorial shows replacing nodes in a Qwen3-8B graph with a TVM-FFI kernel and swapping a DistilBERT region with a CuTe DSL kernel—no TensorRT C++ required. Two paths exist: a recommended path using family-published versioned recipes, and an advanced path with manual graph inspection and node-ID typing. Both share the same backend; DSOs are bound at pipeline construction and can be swapped for ABI-compatible ones without rebuilding the bundle.
Related event: NVIDIA Open-Sources TensorRT Model Connect for Two-Command HF Deployment(2 posts)→
More from Infra
- SK Hynix authorizes massive $29B buyback, betting on non-cyclical AI memory demand — RihardJarc · 2026-08-19
- OpenRouter Cache Hit Rate Soars to 74% Driven by Agentic Workloads — zainhas · 2026-08-19
- Is it safe to run MiniMax on RTX 4060? Overheating issues — ZealousidealCream610 · 2026-08-19
- TSMC Boosts US Investment to $265B for 2nm, CoWoS — ocean_protocol · 2026-08-19
- New paper: 'Intelligence per Watt' measures local AI efficiency — pscoutou · 2026-08-19
- Stable Diffusion Pipeline Deep Dive: Orchestration Over Model Hype — Mahmoud_Zalt · 2026-08-19