How Omnii scales pretraining data mixing to 20+ modalities and thousands of sources
ChengleiSi · x · 2026-10-13
Radical Numerics shares how it scaled data mixing to thousands of sources for pretraining Omnii, its multimodal genome model spanning 20+ data modalities including DNA and natural language. Improved data mixing is described as a key ingredient for large-scale multimodal pretraining; details in the thread.
More from Research
- Harness code becomes a research battleground: CMU rewrites it with RL, Apple says it doesn't matter — burkov · 2026-10-13
- LiveBench 2.0 incoming: agentic long-horizon benchmark you can't benchmaxx — bindureddy · 2026-10-13
- Muse Wonder shows far better OOD generalization than Muse Spark 1.3 — armand_ruiz · 2026-10-13
- Tim Hwang: self-driving research fields in silico are on the edge of reality — CatAstro_Piyush · 2026-10-13
- 546 possible new exoplanets found by mining 4 million stars in TESS data — CatAstro_Piyush · 2026-10-13
- Economists track 213,000 expert hires and $540M paid to train AI — paulnovosad · 2026-10-13