French team drops vLLM/PyTorch for Rust+CUDA and FP4/FP8, huge single-GPU inference gains

IgorCarron · x · 2026-11-18

As the French team speyronnet nears launch of their consumer inference engine ibou, they shared an aggressive optimization push for their in-house short-synthesis model Beaupré: replacing vLLM and PyTorch entirely with Rust + CUDA.

The optimization stack:

The result is a dramatic performance improvement on a single GPU — a textbook case of bypassing general inference frameworks for a hand-rolled extreme-performance stack.

Original post →

More from Infra

Infra channel →