AURORA-LM: A 1B Continuous Diffusion Language Model Trained on Ascend NPU
机器之心 · wechat · 2026-08-08
A joint research team from Nanjing University and other institutes proposed AURORA-LM, a continuous-latent diffusion language model. It achieves strong results in text generation and is fully trained on Ascend NPU, scaling up to 1 billion parameters.
Core Design & Techniques:
- Two-stage Modeling: An autoencoder first maps text into a high-capacity, decodable continuous latent sequence. A block-wise causal diffusion model then learns to generate this sequence.
- Latent Optimization: Text generation quality is significantly improved by narrowing the noisy input dimension (bottleneck) and increasing the probability of high-noise state training.
- Self-Trajectory Consistency: To mitigate error accumulation in few-step denoising, the model introduces a consistency loss that aligns predictions across adjacent denoising states, showing major improvements in few-step generation.
Results:
- At 130M scale, AURORA-LM outperforms autoregressive and other diffusion baselines in both free generation and conditional summarization.
- Scaled to 1B parameters (AURORA-LM-L), it achieves an average score of 32.6 across nine language benchmarks, beating larger continuous language models.
More from Infra
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16
- DeepSeek V4.1 Flash on 4x DGX Spark: weights alone are 511GB, the 494 tok/s claim doesn't fit — Aroochacha · 2026-10-16
- Open-sourced Intel GPU Vulkan inference upgrades: 2x ingestion, 60%+ faster inference — Fun-Aardvark-1143 · 2026-10-16
- ELSFP pain points: >3dB insertion loss, costly PMF, harsh packaging requirements — jwt0625 · 2026-10-16