Claim: Transformers can now be pretrained with zeroth-order optimization, no backprop
teortaxesTex · x · 2026-09-29
@industriaalist claims: "We've figured out how to pretrain transformers with zeroth-order optimization and no backprop," adding that many core assumptions in optimization research are completely wrong. A paper is promised soon; the claim is unverified as of now. @teortaxesTex quote-posted it with a simple "what".
More from Models
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Top LLM score more than doubled since GTA VI trailer dropped — ApprehensiveAd3629 · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- AI daily: DeepSeek rate-limits chats, Qwen haggling spurs shop warnings, Doubao workspace launch — 快鲤鱼 · 2026-11-16