Codex agent shrinks a 27M-param transformer to 11.46M with equal loss via weight sharing
yoshiyama_akira · x · 2026-11-13
- 一位开发者做了一个「agent 架构搜索」实验：给 Codex 一个 27.14M 参数的 transformer,限定允许的修改方式(如 layer loops 等)，要求它在 alpaca 数据集上 5k 步后 val loss 不下降的前提下尽量减少参数。
- Codex 找到了一个古怪的权重共享架构：参数降到 11.46M(减少约 58%),val loss 与 baseline 持平。
- 配图对比了两种架构：baseline 是 8 层独立 Attn+FFN；优化版把 FFN 压缩到约 192 embedding 宽度并循环复用 7 组 FFN 与 5 组 Attn 权重，层间以不同组合重复调用。
- 有趣之处在于方法本身：用编码 agent 替代传统 NAS,靠限定编辑空间 + 训练反馈做架构搜索，复现门槛低。
More from coding & agent
- Gemini /teamwork agents caught live: four parallel task logs all running that second — RyanSpunk · 2026-11-13
- Codex CLI 0.168.0 adds model provider picker, side-by-side TUI workspace and animated GIF generation — github-actions[bot] · 2026-11-13
- Free MCP server lets Claude read Altium, Allegro, KiCad and Gerber PCB files directly — woodisruff · 2026-11-13
- a16z Partner Clones Open-Source Game with Fable 5.2 Agent, Which Finds Its Own Horse Animation Asset — FinanceYF5 · 2026-11-13
- OpenAI team shares how it runs and governs fleets of persistent agents internally — romainhuet · 2026-11-13
- Perplexity's Decider scanned 1B tokens across 7,000 agent sessions, found 14 security fixes — denisyarats · 2026-11-13