Component Benchmark: Hierarchical Model Profiling for Large-scale Recommendation Systems
Dharak Kharod, Yuzhen Huang, Zhou Wang, Jackie Xu, Fuzail Khan, Jacky Zhou, Hao Yan, Lidong Zhao, Xizhou Feng, Yvonne Liu, Karthik Jayaraman, Praveen Ramachandran, Vishwa Karia, Yashasvi Makin
cs.IR
2026-09-25
Meta profiles each nn.Module with live inputs on one GPU; a PT2 opportunity sized at 83→50 ms later delivered +34% QPS, and a 2.8× memory regression mapped to two modules.
Recommendation models have not collapsed onto one Transformer backbone. DLRM, DCN, DHEN, and HSTU mix embedding lookups, small GEMMs, jagged features, and low-arithmetic-intensity ops. Production models are terabyte-scale, run on thousands of GPUs, and ingest on the order of 10¹¹ examples per day. Single-digit throughput gains become fleet capacity.
Existing tools answer at the wrong grain. Nsight and Kineto emit kernel traces. PyTorch Profiler's withmodules attributes ops during a full forward pass, so cache warmth, fusions, and scheduling across modules are tangled; PyTorch 2 compilation then erases module boundaries. Practitioners ask which nn.Module is slow. The tools name a kernel.
Component Benchmark (CB) is about 1,000 lines of PyTorch. Hooks capture each submodule's live inputs at chosen boundaries and profile that module on its own, walking the tree breadth-first. If a parent does not fit on one GPU, CB skips it and aggregates children.
Four plugin layers:
The paper demos on one H100 with open DeepFM-DLRM and NanoGPT. Production models enter through a custom RecSysProvider.
On open DLRM the icicle is the point. sparsearch is almost pure bandwidth. Inside interarch, deepfm sits at 27.06% MFU while its FM-cross sibling is at 0.06%. Two Linear layers in the dense MLP land at 25.91% vs 20.74% MFU, shapes 1024×4096 vs 4096×512. ReLU reports 0 FLOPs and paints red.
On NanoGPT the 12 blocks look alike. Inside one block, attention is about 10% MFU, the MLP about 20%, and layer-norm / GeLU / dropout near 0. Nothing in the framework is recommendation-specific.
Production numbers are usage stories, not controlled experiments:
The conversation moves from "read Perfetto" to "this Linear is at 20% MFU." Heterogeneous recommenders need that grain: one layer is a GEMM, the neighbor is an embedding gather. The same tool runs on LLMs, with less contrast between identical blocks.
This is an internal tooling paper. Open DLRM and NanoGPT numbers are demos, not SOTA comparisons. The production attribution stories are the evidence.
Isolated profiles drop overlap and fusion across modules, so they can over- or under-state real stalls; the GEMV case is exactly a setting where local slowness helped the whole model. One GPU and shrunken embeddings are not a thousand-GPU trainer. There is no error table against PyTorch Profiler withmodules. Production figures lack a protocol, and the paper does not give a public repo. Static icicle figures in the PDF are nearly unreadable; the product is the interactive UI.