LTX-2.5 on one GPU: 130s → 43s per clip by fixing 384 silent fast-attention fallbacks
_BatmanRises_ · reddit · 2026-10-19
A profiling write-up on LTX-2.5 (image+prompt → 10s of 1080p with audio) on an RTX PRO 6000 Blackwell: stock ComfyUI takes 130.1s/19.2Wh per clip, the optimized pipeline 42.9s/6.9Wh with identical output quality.
Key findings and fixes:
- Guide frames silently disable fast attention: the attention mask made SageAttention fall back to the slow path 384× per clip. Running Sage everywhere plus one small exact pass over masked keys, merged exactly, halves attention time.
- Decode+save was 38% (49.7s): tiled VAE decode does 3× the work via overlapping tiles, 43% of decoder time was memory traffic on 17GB tensors. Whole-frame decode, fused CUDA kernels and direct NVENC output cut it to 6.6s, closer to the fp32 reference.
- Transformer: official INT8 + SageAttention nearly halved it; then SageAttention 3 FP4, removing waste (18 full-size copies per layer, a 4.8GB copy per attention call), and block-sparse attention in 41/48 layers (7 layers stay dense).
Discipline: lossless changes must be byte-identical; look-altering changes got side-by-side human review. Code written by Claude, designs reviewed by Codex. Needs 96GB VRAM.
More from Infra
- SonicBoom v0.0.3: compiler-first AI runtime beats llama.cpp decode by 1.18x, calls for GPU benchmarking — nalaginrut · 2026-10-19
- M5 Ultra Mac Studio rumored with 512GB RAM; 6 units chained could run 2T-parameter models locally for ~$100k — SumitGup · 2026-10-19
- Apple Silicon's memory economics: M5 Ultra offers 5.6x better $/GB/s than DGX Spark for LLM inference — SumitGup · 2026-10-19
- Mac Studio M5 Ultra vs Dual DGX Spark: Long-Input Inference Benchmarks Show Split Verdict — GabGarrett · 2026-10-19
- DLite: History-Free Speculative Decoding Fixes 80% Acceptance Drop in Agentic AI — SJTU · 2026-10-19
- Gartner warns $3.5T AI infra spend at impairment risk by 2030; its own stock is down 61% — firstadopter · 2026-10-19