Task Alignment and Magnitude Imbalance Govern Learning Speeds in Linear Networks
ClementineDomi6 · x · 2026-11-05
A paper thread presenting theoretical work on multi-task learning in linear networks. The authors extend the Riccati-based analytical framework to multi-task settings, characterizing how task interactions unfold over training. Their analysis shows how task alignment and relative task magnitudes govern learning speeds and relative task importance, and connects these dynamics to gradient-based interference measures used in practice.
More from Research
- Weight-programming an MNIST MLP with coding agents: near 90% from random init — RobertTLange · 2026-11-05
- Why AI research keeps looking under the lamppost: stale problem frames inherited from Kant — fkasummer · 2026-11-05
- Engram blog: RL training complements text memory, stays robust through millions of token edits — canaesseth · 2026-11-05
- New Book on Generative AI and Stochastic Thermodynamics by Welling Team Out This Month — mjdramstead · 2026-11-05
- New paper uses 288M job postings to measure where AI raises the value of work it can't do — robseamans · 2026-11-05
- Matthew Honnibal: LLMs can 'see' weight matrices — but only one layer at a time — honnibal · 2026-11-05