A Hands-On GPU Profiling Starter Path: Time PyTorch Attention with Stanford CS336 Assignment 2
stanfordnlp · x · 2026-11-22
A beginner-friendly GPU profiling learning path: start by timing a PyTorch attention block at different sequence lengths and recording peak memory, using GPU MODE Lecture 1 for profiling tools and Stanford CS336 Assignment 2 for the model exercise.
The companion repo stanford-cs336/assignment2-systems (MIT license, 384 stars, 829 forks) is the student version of the CS336 "Language Modeling From Scratch" Spring 2026 systems assignment, including cs336-basics and cs336systems modules, the assignment PDF, tests, and even AGENTS.md/CLAUDE.md files.
More from coding & agent
- Gemini 4 Neon builds an interactive 3D neural character demo, no NVIDIA GPU needed — Eric520CC · 2026-11-22
- RLVR trains models to maximize task reward — long-term code maintainability is on you — bendee983 · 2026-11-22
- Free online Lean 4 book packs 23 chapters from BM25 ranking to local LLMs with Ollama — danbri · 2026-11-22
- Claude Code caps Kimi K3.1 at 200K: 148 compactions across 95 trials vs Opus's 797K — zainhas · 2026-11-22
- Engineer says he independently arrived at the same 'knowledge factory' design across domains — hwchase17 · 2026-11-22
- Running Inkling 2 Mini locally on Mac: 20GB model, ~16 tok/s on M4 Pro — jacob_posel · 2026-11-22