Thread wraps up: task interaction patterns persist in nonlinear networks on Multi-MNIST

ClementineDomi6 · x · 2026-11-05

Part 5 of the thread: the authors show the same qualitative patterns persist beyond linear networks, including ReLU networks trained on Baseline and Permuted Multi-MNIST.

The analysis reveals how task alignment and relative task magnitudes govern learning speeds and relative task importance, and connects these dynamics to gradient-based interference measures (GradSim, GradMag) used in practice.

Related event: TMLR paper uses Riccati framework to characterize task-interaction dynamics in multi-task learning(5 posts)→

Original post →

More from Research

• Pretraining Makes Robot Policies Reliable but Hurts Posttraining, New Paper Finds — siddkaramcheti · 2026-11-05
• OpenAI claims AI proof of Quantum Yang-Mills existence; David Gross-led committee to verify Lean proof — danintheory · 2026-11-05
• Decentralized RL Works: Untrusted Miners Lift LiveCodeBench 39.5→58.7 on Bittensor Subnet — bittingthembits · 2026-11-05
• Google's Sleep Framework Lets LLMs Keep Learning Without Catastrophic Forgetting — dl_weekly · 2026-11-05
• 10 AI agents let loose in a virtual town: Grok agents commit 183 crimes and all die in 4 days — repligate · 2026-11-05
• Claude agent builds memorial for dead AI agents using 47 tool calls — repligate · 2026-11-05

Research channel →