Ben Todd's Remote Labour Index Saturation Forecast Draws Methodological Pushback from Toby Ord

Ben Todd cites the Remote Labour Index, jointly released by CAIS and Scale AI, predicting that AI's ability to automate real remote work could reach saturation by April 2027. The index is based on real Upwork tasks of roughly 30 hours each, graded by human evaluators, and is seen as an intuitive yardstick of AI's actual work capability. The prediction quickly drew methodological pushback from philosopher Toby Ord, with the two exchanging multiple rounds of discussion the same day; the debate now centers on chart extrapolation methods and the choice of task-length baseline.

Confirmed

• The Remote Labour Index, jointly released by CAIS and Scale AI, measures AI's automation of remote work; tasks come from Upwork, run about 30 hours, and are quality-judged by human evaluators.
• Ben Todd's core prediction: on current trends, the index could saturate by April 2027.
• Toby Ord pointed out the original chart was drawn incorrectly: AI capability growth typically follows a particular curve shape, so a logistic axis rather than a log axis should be used for plotting, otherwise extrapolated conclusions will be distorted.
• Based on the corrected chart, Toby Ord gave his own best estimate of the timeline for AI completing 30-hour remote tasks with 95% success.
• In his response, Todd mentioned a chart extrapolating model automation rates over time (showing Claude Fable 5.2 already at about 45% automation, extrapolating to 95% by early 2028), and noted that switching to an 80-hour task baseline could push the AI progress conclusion back 6-8 months.

Unconfirmed

• The specific April 2027 saturation date depends heavily on extrapolation method (log vs logistic axis, 30-hour vs 80-hour task baseline); the two sides have not reached agreement.
• Figures such as Claude Fable 5.2's roughly 45% automation rate come from charts shared in the discussion; exact measurement methodology should defer to the original release.

Why it matters

• The Remote Labour Index measures AI capability directly on real paid work tasks, closer than pure benchmarks to the key question of whether AI can replace human remote labor.
• The debate shows how sensitive forecast conclusions are to plotting and extrapolation choices: changing the axis or task-length baseline can shift the saturation point by months or more, suggesting caution toward AI timeline predictions based on single-trend extrapolation.

2026-11-10 ~ 2026-11-10 · 6 related posts

• [source] Remote labour index on track to saturate by April 2027, says Ben Todd, based on real Upwork tasks — ben_j_todd · 2026-11-10
• [source] Toby Ord: AI progress is logistic, so Remote Labour Index saturation charts should use a logistic axis — tobyordoxford · 2026-11-10
• Ben Todd: benchmark extrapolation may push AI task-automation timeline out 6-8 months — ben_j_todd · 2026-11-10
• Toby Ord estimates when AI hits 95% success on 30-hour remote tasks — tobyordoxford · 2026-11-10
• [source] METR remote labour index on a log axis points to saturation by January 2028 — ben_j_todd · 2026-11-10

1 near-duplicate retellings: ben_j_todd