Practical Guide to Training LoRA for Ace Step XL Audio Model

CryptoChangeling69 · reddit · 2026-08-11

The author shares hands-on experiences and pitfalls when training LoRA for the audio model Ace Step XL:

Data Preprocessing: Recommends using MOSS-audio to caption style descriptions and lyrics separately, then formatting them with an LLM into structure tags. Writing Python scripts to run Apollo enhancers and LUFS normalization (-14 loudness) significantly boosts generation quality.
Training Setup: Suggests cutting audio into exact 45-second segments to prevent OOM errors and keep the dataset uniform. The BASE model trains 3x faster than SFT with decent results.
Key Configs:
Ostris AI Toolkit: 183 samples of 45s, LR 0.0001, rank 64, alpha 128, yielding good results at 5500-6500 steps.
Ace Step Gradio UI: 183 samples, LR 0.0001, rank 128, alpha 256, taking about 100 epochs.
Inference Tips: In ComfyUI, SFT models trained via Gradio UI require a high LoRA strength of 1.6 to 2.0 to activate properly. Always fill out Key, BPM, style description, and lyrics completely for a night-and-day difference.

More from Multimodal