Burkov predicts looping recurrent 7B transformers will return and get good at coding
burkov · x · 2026-09-17
Andriy Burkov (author of The Hundred-Page Machine Learning Book) predicts that 7B models built on looping/recurrent transformer architectures with SOTA attention will soon return—and will actually be good for coding. It's a bet on architectural efficiency over scale for small coding models.
More from AGI Musings
- So8res Adds One-Year Retrospective to 'If Anyone Builds It, Everyone Dies' E-book — connoraxiotes · 2026-09-17
- A Year After 'If Anyone Builds It, Everyone Dies', Authors Give Away 1,000 Free E-books — connoraxiotes · 2026-09-17
- 'This will kill everyone, give us more power': blogger mocks recurring AI doom rhetoric from the world's most powerful — kevinnbass · 2026-09-17
- Podcast: TheZhi and Robert Wright debate whether the AI 'slowdown' is real — TheZvi · 2026-09-17
- Ex-HRT quant: LLM agents now let average CS grads do elite quant research — igarciacamargo · 2026-09-17
- Marx's failed predictions mirror today's AI forecasts, argues one observer — kevinnbass · 2026-09-17