Astra Ultra PRs an optimization to karpathy's minGPT, cutting parameter visits 422 to 77
Kuprel · x · 2026-09-10
Kuprel revealed that Astra Ultra submitted a real optimization PR (#161) to karpathy's minGPT: fixing redundant recursive parameter traversal in optimizer setup.
- The setup already walks every module via namedmodules(), but the inner namedparameters() recursively revisits descendants;
- Fix: use recurse=False so each module classifies only its own parameters;
- Result: parameter visits dropped from 422 to 77 on a six-layer test model, with weight-decay groups preserved;
- Validation: on 1/3/6-layer tiny GPT models, optimizer groups (including order) were identical and model states matched bit-for-bit after an AdamW step.
A notable case of an AI coding agent landing a verifiable, mergeable optimization in a well-known open-source repo.
More from coding & agent
- Instinct launches Trusted Person network letting AI agents negotiate plans with each other — mon__lim · 2026-09-10
- diagram-design skill gives Codex and Claude Code 39 actually-good diagram types — daniel_mac8 · 2026-09-10
- Simular to host CUA party at SF Tech Week debating if 2027 brings AGI for computer-use agents — xwang_lk · 2026-09-10
- LangChain's Harrison Chase: code-writing LLMs will make computer use take off — hwchase17 · 2026-09-10
- Open-source Chrome extension lets a browser agent keep working in its tab while you browse elsewhere — Silly_Entertainer92 · 2026-09-10
- Letting GPT-6 Astra drive software UIs directly beats built-in agents, dev reports — brandon_galang · 2026-09-10