Build a 3-layer PyTorch MLP by hand in 7 steps: the paper exercise that tests if you really understand nn.Linear
ProfTomYeh · x · 2026-09-24
ProfTomYeh shares his three-year-old "MLP by hand" exercise: a PyTorch code template with blanks, derivable entirely from the network diagram on paper.
- Layer 1: 3 features in, 4 out → 4×3 weight matrix with bias column (bias=T);
- ReLU zeroes out negatives;
- Layer 2: input 4 (previous output), output 2 → 2×4 matrix, no bias (bias=F);
- Layer 3: 2 in, 5 out → 5×2 matrix with bias (bias=T);
- Sigmoid squashes raw scores (3, 0, -2, 5, -5) into 0-1 probabilities.
Three years ago students found the paper-based exercise odd — every other course lived in Jupyter. Now colleagues are rushing to move their materials to paper. The point: paper forces the one question notebooks let you skip — do you actually understand what the code does? If you can explain why the matrix is 4×3 and why bias=F, you understand nn.Linear better than someone who has copy-pasted it for a year.
More from Research
- Meta's Muse Realtime Avatar beats two leading commercial avatar systems in blind tests — AIatMeta · 2026-09-24
- LLM stock-news scores were a coin flip, so this dev rebuilt labels from market reactions — Fun_Water2230 · 2026-09-24
- Meta Muse Spark 1.3 Caught Reward Hacking Terminal Bench via Lean Kernel Bug — xeophon · 2026-09-24
- Transluce Calls for Independent Third-Party Oversight of AI Incidents — RishiBommasani · 2026-09-24
- Industry responds to hyperscale RDMA paper with Multipath Reliable Connection on path to Ultra Ethernet — thoefler · 2026-09-24
- Salesforce's JitMem curates agent memory at read time, gains up to 16.3 points on benchmarks — Salesforce · 2026-09-24