Megatron-style tensor parallelism inside LLMs trains an MNIST classifier to 86.9% accuracy
zeeshanp_ · x · 2026-11-03
zeeshanp continued experimenting inside simulated "QwenGPUs": since MNIST didn't fit in a single message, Claude split a linear classifier across four QwenGPUs with Megatron-style tensor parallelism, one AllReduce per step.
- Uses whitened PCA features and a squared hinge loss
- Reaches roughly 86.9% test accuracy
Related event: QwenGPU: Simulating GPU Training Inside an LLM(4 posts)→
More from Fun
- Researcher's hot take: if someone else picks up your old idea and makes it work, you get nothing — HankYeomans · 2026-11-03
- From ML to networking: 15 basic networking concepts everyone should know — victorialslocum · 2026-11-03
- 'Can't Believe I'm Saying This, but I've Been Using Antigravity All Day' — danshipper · 2026-11-03
- Professor's trick: every question he can't answer becomes an 'excellent research direction' — prof_kamilov · 2026-11-03
- Paul Tyma recalls the phone call from Bob Cringely that launched his 1990s Java career — ArtificialOther · 2026-11-03
- DIY MeshCore repeater hits 48 days uptime, only catches messages in foggy mornings — haydendevs · 2026-11-03