NUS researcher Xue Fuzhao: loop transformer ideas are cheap, careful execution is the AGI route
XueFz · x · 2026-09-02
Xue Fuzhao revisits his loop transformer work: Looped MoE Transformer ('Go Wider Instead of Deeper') shares parameters across depth and replaces FFN with MoE across width, beating ViT on ImageNet-1K by 1.5% with 0.72× params and ViT-MoE by 2.1% with 0.13× params. But he says these obvious, cheap ideas matter little — careful, sound execution and deep understanding is the route to AGI.
More from Research
- NASA-IBM AI model maps Asian rice paddies at 30m resolution to track water and methane — anselm · 2026-09-02
- By LeCun's math definition of extrapolation, everything a neural net does is extrapolation — burny_tech · 2026-09-02
- GPT-5.6 Solves Decades-Old Information Theory Conjecture in Yale Professor's arXiv Paper — burny_tech · 2026-09-02
- Google Research's TimesFM time-series foundation model draws renewed GitHub attention — google-research · 2026-09-02
- Most open-source AI text detectors can't hold a 0.5% false-positive rate — grumpyp2 · 2026-09-02
- Preference models to triage AI research ideas win praise from DeepMind's Edward Hughes — j_foerst · 2026-09-02