Discussion on Scaling Laws: Parameters, Depth, and Test-Time Compute
samsja19 · x · 2026-08-20
A technical discussion responding to thoughts on scaling laws:
- Scaling Strategy: Targets for scaling active vs. total parameters independently.
- Depth Trade-offs: Balancing model depth, active parameters, and test-time compute.
- MoE Architecture: Whether to maximize expert count if memory and EP bandwidth on NV72 are sufficient.
More from Infra
- Hope to live long enough to see everything become data centers — zck · 2026-08-20
- View: If TerraFab succeeds, compute will be cheap again — teortaxesTex · 2026-08-20
- Prediction: Models Will Grow Larger, Chips Won't Get Cheaper — teortaxesTex · 2026-08-20
- On-Chain Data Needed for Managing Millions of Parallel Agent Threads — curious_vii · 2026-08-20
- If internet preceded home computers, we'd likely still be on mainframes — maxsloef · 2026-08-20
- DFlash2 Quantization: Qwen3.8 27B on 2x3090 with 262k Context — luedtek · 2026-08-20