Microsoft MAI reveals in-house 3T-parameter base models with native encoder-free VLM pretraining
burny_tech · x · 2026-10-19
Mustafa Suleyman announced MAI has built SOTA pre-trained base models in the 3T-parameter weight class entirely in-house, with post-training results coming soon.
The technical blog detail (screenshot) shows MAI-Base-2 pre-trains natively with vision using an encoder-free VLM architecture:
- VLM corpus of 4.5T unique tokens spanning chart understanding, grounding, spatial reasoning, medical, computer use, sheet-music reading; VLM data made up 18% of the 50T training mixture with repetition
- vs MAI-Base-1 (30T): code +34%, web +16%, PDFs +104%, STEM +24%, books +194%, math +37%, vision-only +22% (14% share)
SenseNova (liuziwei7) echoed the paradigm, pointing to its open-source NEO and SenseNova-U native VLM series with papers, weights and training/eval code released.
Related event: Microsoft reveals 3T-parameter MAI foundation model(2 posts)→
More from Models
- Musk endorses Grok @Bot: 'like hiring an incredibly competent employee' — elonmusk · 2026-10-19
- Security researchers hit cyber guardrails mid-incident; debate over who gets 'cyber models' — Bedrovelsen · 2026-10-19
- Redditor impressed by MiniMax H3: "It nailed this one" — Cheap_Credit_3957 · 2026-10-19
- DeepSeek V4.1 gets smarter but discards its best ideas; Opus struggles to rescue the mess — teortaxesTex · 2026-10-19
- Same agent build task: DeepSeek costs $0.21 vs Opus 5.5's $2.20 — teortaxesTex · 2026-10-19
- Gemini 4 Argon's real wins: 6M lines stripped from Chromium, C++ to Rust at 2.7x speed — ITower__Education · 2026-10-19