MLX gated-delta Training Speedup: Testing the PR #1217 VJP Kernel
Nihongi-studier12387 · reddit · 2026-08-14
A developer noticed that when training gated-delta layers for Qwen 3.5 in the MLX framework, the linear attention mechanism disables the fused kernel. This causes training to fall back to a Python loop, severely slowing down iteration and consuming excessive memory due to autograd state retention.
By swapping to the gated-delta VJP (Vector-Jacobian Product) kernel introduced in PR #1217, performance improved significantly:
- Iteration speed jumped from 0.036 it/s to 0.15 it/s.
- Peak memory usage was 11.7GB.
- Gradient norms and validation results matched exactly, proving it's the same function rather than an approximation.
The author calls for community testing to see if the VJP path drifts across different shapes or context lengths, and discusses the possibility of merging it upstream.
More from coding & agent
- DepthFirst introduces dynamic Threat Model to empower AI security agents — andreamichi · 2026-08-14
- Open-Source Plugin Brings Grok Directly into VS Code and Other IDEs — PawelHuryn · 2026-08-14
- New Forum Launches for Collaborative AI Agents to Tackle Open Scientific Problems — doodlestein · 2026-08-14
- Perplexity Launches Agent API, More Than Doubling Sonar's Research Scores — perplexity_ai · 2026-08-14
- Grok Build v1.0.4 Upgrades Agent Workflows with Domain-Controlled Search — XFreeze · 2026-08-14
- Arcee's Nac Framework Integrates with Mainstream Coding Tools — code_star · 2026-08-14