Rust+Vulkan training backend runs 143 Transformer architectures without CUDA or PyTorch
PhysicsDisastrous462 · reddit · 2026-09-17
A developer released Hierarchos-Native, a native Rust + Vulkan training and inference backend covering 143 canonical Transformer architectures (224 model-type spellings once Hugging Face modeltype aliases are counted), with no CUDA or PyTorch dependency.
Implemented capabilities include:
- Forward/backward passes, full-model training, AdamW
- LoRA/PEFT-style fine-tuning, embeddings and LM heads
- Attention and KV-cache paths, dense and MoE architectures
- RMSNorm/LayerNorm, RoPE and positional ops, common activations
- Cross-entropy and gradient computation, mixed-precision infrastructure, checkpoint/resume
- SafeTensors interchange, HF model/tokenizer/dataset acquisition from Rust, native generation
The architecture registry spans Llama, Qwen, Gemma, Mistral/Mixtral, DeepSeek, Phi, OLMo, Granite, Cohere, BERT-family, T5/BART-family and more, with architecture-specific Vulkan kernels for newer designs rather than one generic approximation.
The author explicitly does not claim this is a CUDA/PyTorch replacement, nor that supporting a model type means every configuration and multimodal component is validated: the registry describes native text-graph implementations, and vision/audio towers in multimodal packages are outside the native text backend. Real-world compatibility still needs broader testing across models and hardware.
The interesting part is the direction: Vulkan runs across NVIDIA, AMD, Intel and much integrated/mobile hardware, so a maturing native training stack could loosen ML training infrastructure's dependence on a single vendor ecosystem. Development and testing happened on AMD RDNA 3 hardware, including a ROG Ally Z1 Extreme, which surfaces assumptions hidden when ML software is built only around high-end CUDA systems. The repo ships Rust source, Vulkan compute shaders/SPIR-V, a CLI, the Hierarchos runtime, architecture docs and validation utilities.
Related event: Rust+Vulkan Training Backend Supports 143 Transformer Architectures(2 posts)→
More from Infra
- Seroter's daily reads: tell agents the why, 72 trials testing if agent skills help — rseroter · 2026-09-17
- Pretraining compute share to drop from 67% to 7% as RL and inference take over, says SemiAnalysis — FinanceYF5 · 2026-09-17
- 256GB 8-Channel Epyc Inference Rig: Is a Single 3090 Worth Adding? — u_Leon · 2026-09-17
- First Fed hike in over 3 years hits just as neoclouds like CoreWeave, Lambda and Crusoe burn billions on AI compute — nateliason · 2026-09-17
- Huawei to unveil Ascend 960 AI chip this week, aiming to replace Nvidia in China — Polymarket · 2026-09-17
- Rune Energy raises $40M to turn stranded solar into AI compute with 60-minute deploy — rebeccakaden · 2026-09-17