CUDA+ROCm dual-backend LLM inference on mixed-vendor GPUs, up to 48% faster
Ren-WuJun · reddit · 2026-11-10
A developer forked Project MAYA to run GLM 5.3-Flash cooperatively across an RTX 5090 (CUDA backend: attention, dense/shared layers, expert cache) and a Strix Halo iGPU (ROCm backend: complementary experts during prefill and decode), achieving up to 48% speedup. The 3.7k-line PR has been welcomed by the Project MAYA author for testing toward a merge.
More from Infra
- GB300 Rents for $12-15/hr as Inference Supply Outstrips a Thin Buyer Market — teodorio · 2026-11-10
- Meta FAIR's e-means beats k-means in 1/10 epochs with 10-45x GPU speedups — burny_tech · 2026-11-10
- Report: NVIDIA May Cut TSMC Orders as AMD and ASIC Makers Circle for Capacity — zephyr_z9 · 2026-11-10
- RTX 5090 local LLM math: a day of inference outputs less than $1 worth of API usage — PurzBeats · 2026-11-10
- CoreWeave needs ~350MW more live by year-end — the exact spot it slipped last year — tengyanAI · 2026-11-10
- Memphis data centers led by xAI generated $54M in tax revenue amid pause proposal — pickover · 2026-11-10