Halcyon: no text pretraining, beats LlamaGen-XL/8 with 18× less compute
JosephJacks_ · x · 2026-10-16
Radical Numerics details Halcyon (2/8): a multimodal architecture with no text-only pretraining and no frozen language models. At 0.1-4B parameters trained on 100M image-text pairs from GPIC, it outperforms strong late-fusion baselines. Counting the baselines' LM pretraining compute, the 0.52B model beats LlamaGen-XL/8 with 18.2× less compute, and the 1.6B model beats JiT-T2I-XXL/8++ with 275× less.
Related event: Halcyon: Text-to-Image Base Models That Cut Pre-training Compute 275x(3 posts)→
More from Multimodal
- Meta's Alexandr Wang shows Muse video model directing and editing an entire cinematic shot — alexandr_wang · 2026-10-16
- Krea 3 image editing model lands on fal: high-quality edits in under 6 seconds — gorkem · 2026-10-16
- Tripo P2 vs Meshy 7.1: benchmarking today's best image-to-3D models for game assets — noahsolomon · 2026-10-16
- BytePlus-backed AI Film Award offers $1M grand prize for films 70% made with Seedance — DavidmComfort · 2026-10-16
- Halcyon: from-scratch early-fusion T2I models match late fusion with 275x less pretraining compute — StanfordAILab · 2026-10-16
- FLUX 3 Image Klein: open-weights image model runs on 8 GB consumer GPUs — The Decoder · 2026-10-16