The S-Curve Debate: Can AI Progress Keep Stacking New Paradigms

From August 28–30, researchers including kuza55, Jules Jacobs, Herbie Bradley and Dylan Bowman held a multi-thread discussion on the sustainability of AI progress. They broadly agree that current methods should keep working for a while, but since all technologies follow S-curves, whether progress can be sustained by stacking new S-curves remains unresolved, making this a debate worth tracking.

Confirmed

• kuza55 argues software engineering will clearly be solved and progress is fast, but trajectories outside software/computer use/math look weaker
• On the height of the current S-curve's endpoint, Jules Jacobs asked whether it exceeds "building a better C compiler than GCC/LLVM"; kuza55 said it is higher and "we're close," while Jacobs noted this has never been achieved before
• Jules Jacobs contends not everything needs to be an RL environment—only AI development itself; kuza55 raises two bottlenecks: running out of compute funding and running out of data
• On robotics, Jules Jacobs asked whether an automated AI factory could autonomously train models for RL-able tasks like robotics; kuza55 sees trajectories outside the RL-environment box as clearly weaker, requiring sim2real or RL-efficiency breakthroughs, while Jacobs counters that an "RL factory" with many robots plus improving general LLMs for human-like diagnosis is not implausible
• Herbie Bradley notes the difficulty of LLM-as-judge for long tasks lies not in length but in capturing finer qualitative judgments, largely dependent on pretraining scale and data coverage; he also argues that purely lengthening RL horizons without changing pretraining makes capabilities more uneven rather than smoother, and RL sample inefficiency is a challenge
• Herbie Bradley and William A. Roberts discuss the underappreciated implications if AI far exceeds humans in verifiable domains (math, code) yet still cannot run a business after another year
• Dylan Bowman proposes continual learning or longer RL episodes could absorb ambiguous, unstructured problems, and that code quality can be learned implicitly via long-horizon, more ambitious tasks rather than finer-graded scorers; Bradley and kuza55 remain skeptical about the design space being too large and the fundamental goal-definition problem

Unconfirmed

• Both sides admit uncertainty about how long S-curves can be stacked; whether robotics training is stuck within the LLM S-curve remains contested
• Whether continual learning and longer RL horizons can close the gap in low-verifiability domains is unresolved, with opposing views

Why it matters

• The discussion turns "will AI keep progressing" into testable questions—can software engineering be solved, can AI beat GCC/LLVM—providing yardsticks for tracking progress
• Making AI development itself an RL environment is a potential route around data and compute bottlenecks
• If the asymmetry between fast verifiable domains and slow low-verifiability domains persists, it could reshape business and society; the robotics debate maps both the paradigm's boundary in the physical world and an alternative path via RL factories plus general LLM diagnosis

2026-08-28 ~ 2026-08-30 · 16 related posts

• What if AI becomes superhuman at math/code but lags in business? — williawa · 2026-08-28
• [source] If AI becomes superhuman at math but still can't run a business — herbiebradley · 2026-08-28
• Jules Jacobs: you only need AI development itself as an RL environment — JulesJacobs5 · 2026-08-29
• [source] S-Curve View: Uncertainty on Stacking AI Paradigms — kuza55 · 2026-08-29
• Are we hitting limits? AI S-curves and scaling uncertainty — JulesJacobs5 · 2026-08-29
• AI coding S-curve peak: Better than GCC/LLVM? — kuza55 · 2026-08-29
• Software engineering will be solved, but AI struggles outside software/math — kuza55 · 2026-08-29
• If software gets solved, could an automated AI factory self-train robotics models? — JulesJacobs5 · 2026-08-29
• Robotics may be stuck inside the LLM S-curve without sim2real or RL efficiency breakthroughs — kuza55 · 2026-08-29
• Robotics breakthroughs need Sim2Real or RL efficiency leaps off the LLM curve — kuza55 · 2026-08-29
• Continual learning or longer RL horizons as solutions — dylanbowmanSF · 2026-08-29
• RL factory + LLMs could drive robotics progress — JulesJacobs5 · 2026-08-29
• [source] LLM judge quality hinges on pretraining scale; full online RL alone won't crack continual learning — herbiebradley · 2026-08-29
• Scaling RL length alone may increase jaggedness — herbiebradley · 2026-08-29
• Code Quality Learned via Long Horizon RL, Not Granular Graders — dylanbowmanSF · 2026-08-29
• Technical View: Large design space limits RL in automating high-level judgment — herbiebradley · 2026-08-30