A Hands-On GPU Profiling Starter Path: Time PyTorch Attention with Stanford CS336 Assignment 2

stanfordnlp · x · 2026-11-22

A beginner-friendly GPU profiling learning path: start by timing a PyTorch attention block at different sequence lengths and recording peak memory, using GPU MODE Lecture 1 for profiling tools and Stanford CS336 Assignment 2 for the model exercise.

The companion repo stanford-cs336/assignment2-systems (MIT license, 384 stars, 829 forks) is the student version of the CS336 "Language Modeling From Scratch" Spring 2026 systems assignment, including cs336-basics and cs336systems modules, the assignment PDF, tests, and even AGENTS.md/CLAUDE.md files.

Original post →

More from coding & agent

coding & agent channel →