Zhipu chief scientist reveals GLM-5.5 training pipeline: 3 sizes, 6 internal data engines
AccBalanced · x · 2026-11-06
Jie Tang revealed GLM-5.5's training pipeline (32B/122B/256B) in a Tsinghua lecture. Instead of public datasets, pretraining draws from six internal engines: 1T web pages crawled at 10B pages/day with PB-scale template clustering, 2B scientific PDFs parsed via GLM-OCR (4x faster with speculative decoding), 50M books with layout reordering and garbled-text filtering, 500M repos converted into executable tasks with real tests, and 10B videos for long-sequence training. Shared via a tongue-in-cheek Primeagen quote about data scraping.
Related event: Zhipu's Tang Jie Reveals GLM-5.5 Training Pipeline and Data Roadmap(4 posts)→
More from Models
- FLUX 3 models beat FLUX.2 in all 19 Text-to-Image categories; Klein tops $9/1k price frontier — ArtificialAnlys · 2026-11-06
- Anthropic chief architect Sam McCandlish's first post hints at internal-acceleration report, possibly hitting 2x RSP threshold — inductionheads · 2026-11-06
- 6,662 decisions benchmarked: decision model Mammoth-1 averages 80.9%, beats best small LLM by ~3 points — pournasserian · 2026-11-06
- Hesamation: OpenAI Models Beat Anthropic's Only at Certain Kinds of Games — Hesamation · 2026-11-06
- Meta's new 'walrus' checkpoint reportedly matches crouton-alpha quality — ChrisGPT · 2026-11-06
- Quick test: Gemini 3.1 Pro knows the answer in its weights, Flash doesn't — giffmana · 2026-11-06