2D-RoPE for image models is easier in the complex representation, says arohan
_arohan_ · x · 2026-09-05
Replying to @thomasahle, arohan suggests generalizing RoPE (rotary position embeddings): image models can use a generalized form like 2D-RoPE, which is easier to implement in the complex representation. A brief technical take with no further details or experiments.
More from Research
- Open-Source Notebooks Implement RL Algorithms From Scratch, From Q-learning to PPO — ZabihullahAtal · 2026-09-05
- 60+ annotated deep learning paper implementations in PyTorch, updated weekly — ZabihullahAtal · 2026-09-05
- AI2 releases climate emulator for E3SM that captures rainfall up to 99.99th percentile — allen_ai · 2026-09-05
- AI climate emulator holds steady over 400 years, beating E3SM's own bias — allen_ai · 2026-09-05
- AI2's coupled climate emulator SamudrACE runs 1,100 years/day on one H100 vs E3SM's 28 on 105 nodes — allen_ai · 2026-09-05
- AI2 and E3SM Team Unveil SamudrACE, Coupling Atmosphere and Ocean ML Emulators — allen_ai · 2026-09-05