FP16 vs FP32 matmul on RTX 4060: 3.5x speedup at large sizes, slower at 128x128
cneuralnetwork · x · 2026-10-23
A PyTorch benchmark on an RTX 4060 laptop GPU (TF32 disabled, CUDA event timings) shows FP16 matmul is slightly slower than FP32 at 128x128 (0.93x), but speedup climbs with size — 2.21x at N=512, 3.47x at N=1536 — then plateaus around 3.3-3.5x up to 16384x16384. Takeaway: half precision doesn't automatically mean 2x faster; kernel selection, hardware utilization, and whether the workload is large enough to hit specialized compute units dominate. The full curve includes latency plots for both precisions.
More from Infra
- GitHub moved 99.99999% of repos to Azure; per-reference quorums unlock agent-scale Git concurrency — marlene_zw · 2026-10-23
- Jensen Huang visits Astana, joins Kazakhstan's AI minister to discuss nation's AI future — nvidia · 2026-10-23
- BaseQuant V1 quants for agentic Qwen3.8-27B hit 17-34% lower KL than Unsloth — nightowlsleeping · 2026-10-23
- PhD students in some countries get B200s on demand, unlike their US peers — HildeKuehne · 2026-10-23
- Nvidia's Berlin GTC spotlights humanoid robots and Vera Rubin AI supercomputing — emmanuelvivier · 2026-10-23
- Lenovo adds 5 AI Library use cases, claims 1-week deployment and up to 70% lower token cost — shashib · 2026-10-23