NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech
Qiaolin Wang, Pedro Sandoval-Segura, Anunaya Joshi, Edvardas Jurkonis, Jake Downie
cs.SD, cs.AI, cs.CL, eess.AS
2026-09-26
NVAlign backprops NV-ASR tag probabilities through a two-step flow surrogate; English production TTS human tag-following rises from 39.8% to 54.0%, above SFT and Flow-GRPO.
A laugh, a sigh, or a gasp changes how a spoken line is heard. Today's TTS systems already take inline NVV tags such as [laughs], yet uncommon sounds have little labeled audio. After supervised fine-tuning, models still omit tags or emit the wrong vocalization.
Continuous autoregressive flow-matching (AR-FM) TTS is a poor fit for the usual post-training toolkit. An autoregressive transformer reads text and prior acoustic patches and emits a continuous conditioning vector. A diffusion transformer (DiT) flow head then generates the next acoustic patch with flow matching. Discrete-token TTS can run GRPO on token likelihoods. Continuous samplers do not. Flow-GRPO moves likelihood-ratio updates onto stochastic differential equation sampling, and in these experiments it still trains only the flow head. Bland AI tests a direct alternative: freeze an NVV-aware ASR, take its target-tag probabilities as a differentiable reward, and push those gradients back through generation.
Public training mixes 834.5 hours from six NVV corpora into a 39-tag inventory, with benchmark text and speaker overlap removed. Both the TTS model and an NV-ASR based on Qwen3-Omni-30B are fine-tuned so the recognizer learns full transcripts that include tags. Only 18.9 hours of that mix are English, so the production study uses a separate internal corpus and a separate NV-ASR. Mandarin runs use VoxCPM2 and dots.tts; English uses a production model with the same AR-FM layout.
During post-training the NV-ASR is frozen. Generated audio is scored under teacher forcing. For a multi-token tag the reward takes the minimum token probability, then sums across tags in the utterance. The min forces every piece of the tag to be recognized; the sum allows several vocalizations in one line.
Those rewards have to travel through the flow-matching sampler. Full-trajectory backprop is expensive, so NVAlign uses LeapAlign's two-step gradient surrogate: the forward pass follows the sampled path, the backward pass evaluates the DiT at two flow times k>j, and a straight-through connector keeps the forward value equal to the sampled state while routing gradients through the estimate. Gradients go through a frozen vocoder into the flow head, then through the conditioning vector into the backbone. Only LoRA adapters are trained, with ranks 16, 32, and 64 on the backbone, flow-head input projections, and flow head.
Raising tag likelihood alone wrecks the voice. Fidelity penalties use a frozen ECAPA-TDNN for speaker identity, a frozen UTMOS model for predicted MOS, plus peak and low-level waveform terms. Reference-velocity regularization pins the current velocity field to the SFT model. Drop that regularizer and the NVV reward rises from 0.468 to 0.488 while Gemini accuracy/perceptual effect falls from 3.54/3.34 to 2.88/1.94. A higher reward can sound worse.
Each language has a 100-clip listening set (Mandarin covers all 39 tags). Three blinded raters must all agree that the target sound occurs at the marked position. Flow-GRPO starts from the same SFT checkpoint and updates only the flow head.
| System | SFT | Flow-GRPO | NVAlign |
| VoxCPM2 | 43.2% | 37.8% | 47.6% |
| dots.tts | 63.2% | 61.1% | 65.4% |
| English production TTS | 39.8% | 31.8% | 54.0% |
On the NVV-SuperBench human subset, VoxCPM2 moves from 27.4% to 34.9%, about 15.8 points above Flow-GRPO at 19.2%. dots.tts falls from 57.1% to 54.4%. Whisper tag recovery and SED detection rise on all three systems. Paired 95% bootstrap intervals exclude zero for both English gains over SFT, both Mandarin VoxCPM2 gains over Flow-GRPO, and the English 100-text gain over Flow-GRPO. Every other comparison includes zero.
SIM, UTMOS, and DNSMOS all rise, but SIM and UTMOS sit inside the training objective. Human naturalness MOS stays within 0.1 of SFT. Mandarin CER rises 1.26 points on VoxCPM2 and 0.13 on dots.tts; English WER rises 0.07 on the listening set and 1.33 on SuperBench. English SuperBench human tag-following goes from 33.7% to 53.7%. Mandarin Gemini accuracy falls (VoxCPM2 3.36 to 3.24, dots.tts 4.09 to 3.76), and English perceptual effect falls as well.
Flow-head-only NVAlign already reaches an NVV reward of 0.441, above Flow-GRPO at 0.392 and SFT at 0.397. Strip the speaker penalties and SIM collapses from 0.796 to 0.360. The NV-ASR reward does not police those properties.
On continuous AR-FM TTS, SFT still misses tags, and Flow-GRPO often scores below SFT on listening sets. Backpropagating a recognizer through the sampler is a post-training path that actually runs on this architecture, and LoRA is enough.
The gain is incremental. The English production jump from 39.8% to 54.0% is the large one. Public Mandarin models move less, and dots.tts regresses on the SuperBench human subset. The useful setting is narrow: NVV labels exist, the model is already continuous AR-FM, and the goal is whether the tagged sound is actually produced at the marked position. Expressiveness and quality do not rise in lockstep.
Sparse labels for rare NVVs also starve the NV-ASR reward. Optimizing that learned score can exploit recognizer shortcuts and damage prosody. Stacking penalties turns gradient balance into part of the problem. The objective assumes discrete events, so continuous or overlapping vocalizations are out of scope. That list is the paper's own.
Listening tests ran through Podonos with no institutional ethics review. Most Mandarin confidence intervals include zero, so the three systems are not a clean sweep. Gemini 2.5 Pro accuracy and perceptual effect fall in several settings; SIM and UTMOS gains cannot stand as independent evidence because they were optimized. Public English data is only 18.9 hours. The production 14.2-point jump uses internal data and an internal training setup, so outsiders cannot split data, model, and method.