Marin releases open scaling suite Delphi; author's revised prediction cuts error to 0.4%

The Marin team (author dlwh) released Delphi, the first fully open scaling suite, including a scaling recipe, a family of models trained on the TPU Research Cloud, and a scaling law method that extrapolates large-model loss from small models—claiming that extrapolating 300x (starting from 3e17 FLOPs) yields a prediction error of only about 0.2%. dlwh then publicly corrected a previous scaling extrapolation for Hero pretraining: the initial version used the Uncheatable dataset convention, fixing the asymptote E at 1.4 and counting output-head FLOPs, whereas the Paloma convention excludes the output head—which at the smallest compute scale accounts for roughly 65% of total compute, but has a smaller effect at the 535B scale. After switching to the Paloma convention and fitting the loss floor using small-scale experiments, E came out to about 1.20, reducing overall prediction error by about 79%.

Confirmed

• The Delphi suite is open-sourced, including the recipe, models, and scaling law method
• The fitted formula in simplified form is loss = E + A·compute^(-alpha); the hard part is estimating the asymptote E at the small-model stage, and dlwh also revealed that part of the actual fitting work was done by Codex
• Two versions of the prediction were compared: for the same Hero 535B-A23B run, the initial Uncheatable-convention prediction had an RMS error of 1.9%, while the Paloma-convention prediction had an error of 0.4% (RMSE 0.00856)
• dlwh acknowledged the correction was done in hindsight, but stressed that the method matches the earlier Paloma fit and did not use any loss data from Hero 535B; code and context are open-sourced on GitHub (marin-community/marin-logbooks issue #8)

Why it matters

• This is among the first fully open validations of scaling law extrapolation: convention differences (whether output-head FLOPs are counted) can cause compute-accounting deviations of up to 65% at small scales, directly affecting extrapolation conclusions
• It demonstrates the feasibility of extrapolating large-model loss from small models (0.2%0.4% error), with the entire pipeline reproducible
• dlwh self-deprecatingly concluded that the problem wasn't the prediction itself but failing to do a proper QA check before posting, reflecting the full predict–correct–review cycle of open research

2026-10-27 ~ 2026-10-27 · 8 related posts

• [source] Marin Shares Forecast-vs-Measured Plots: Paloma Hits 0.4% RMS Error — dlwh · 2026-10-27
• [source] Marin's Open Scaling Law Extrapolates 300× and Predicts Loss Within 0.2% — dlwh · 2026-10-27
• Marin's dlwh Explains the Scaling Law Fit: Estimating the Loss Floor Early — dlwh · 2026-10-27
• Delegating scaling-law asymptote fitting to Codex while planning a birthday party — dlwh · 2026-10-27
• [source] Fixing compute convention and fitted floor cuts scaling forecast error ~79% — dlwh · 2026-10-27
• Retrospective: Hero scaling forecast fixed without using Hero losses, code open-sourced — dlwh · 2026-10-27
• Scaling-fit discrepancy: output-head FLOPs are ~65% of compute at the smallest rung, ~3% at 535B — dlwh · 2026-10-27
• 'We're not bad at forecasting—I'm bad at QA-ing my posts,' dlwh wraps up — dlwh · 2026-10-27