Poolside details how synthetic data and strict checks helped train a 118B coding model

AI Engineer · youtube · 2026-07-25

Poolside says synthetic data is now a core part of pretraining for coding models

Marah Abdin and Robert McHardy describe how poolside manufactures more code data once high-quality human code runs out, but keeps the generation process difficult enough to teach the model instead of making tasks trivial.

Their pipeline uses templates plus extra context, then varies phrasing and difficulty so the model sees a broad but useful distribution of tasks.
Existing data is also transformed into multi-stage outputs: character swaps, plot changes, and single prompts turned into multi-turn chats.
An orchestrator filters every generation and discards outputs that miss the target.
For training, they use strict reproducibility checks: two replicas on the same data must produce the same number or the run is killed.
That discipline exposed several scale bugs, including broken GPUs, a tensor-parallel precision issue, and silently corrupted gradients from a race condition.
The team says the recipe has held up at scale and produced a 118B-parameter model for agentic coding whose early results already edge out GLM 4.5 Air.

Original post →

Poolside details how synthetic data and strict checks helped train a 118B coding model

More from coding & agent