ByteDance's looped language models match 12B rivals at 1.4B size, with Bengio as co-author

peterjliu · x · 2026-09-03

ByteDance open-sourced Ouro, a family of pre-trained Looped Language Models (LoopLM) that build reasoning into pre-training via latent-space iterative computation, an entropy-regularized depth-allocation objective, and 7.7T tokens of scaling. Ouro 1.4B and 2.6B match up to 12B SOTA LLMs across benchmarks, with gains attributed to superior knowledge manipulation rather than capacity — and reasoning traces more aligned with outputs than explicit CoT. Authors include Yoshua Bengio; the work is cited alongside OpenAI's reduced CoT monitorability as an instance of unauditable latent reasoning.

Original post →

ByteDance's looped language models match 12B rivals at 1.4B size, with Bengio as co-author

More from Models