SymphonyGen: 3D Hierarchical Orchestral Music Generation with Controllable Harmony

petewoodbridge · x · 2026-08-05

SymphonyGen is a 3D hierarchical framework for contemporary orchestral music generation. It uses cascading decoders to decompose bar, track, and event axes, significantly reducing decoding memory while enabling conditioning at every structural level.

The model introduces a beat-quantized multi-pitch harmony skeleton for conditioning, allowing users to generate rich orchestral textures via outline control. It is refined using reinforcement learning (GRPO) with a cross-modal acoustic reward from CLaMP 3 audio embeddings, and employs a dissonance-averse sampling algorithm during inference. Evaluations show that these mechanisms effectively reduce dissonance while maintaining melodic independence, achieving higher subjective quality and preference than baseline systems.

Original post →

More from Multimodal

Multimodal channel →