Is T4 Really 170x Slower Than A100?
Future-Structure-296 · reddit · 2026-07-15
While running a point-tracking model, the author found the NVIDIA T4 to be roughly 170x slower than the A100: processing half a video took 0.5 seconds on the A100 and 85 seconds on the T4.
The model settings included: 47 frames, 256×256, batch=1, pure FP32, with an architecture featuring local 4D correlation volume and a transformer for temporal context. The author ruled out common issues like disabled GPUs, models not loaded onto the card, ineffective cudnn.benchmark, and slow performance across two separate T4 machines. They suspect the bottleneck is related to the architecture or operator characteristics and plan to start with profiling to locate the issue.
More from Infra
- vLLM eyes Rust migration as inference engines race to rewrite the stack — AccBalanced · 2026-10-27
- Anthropic's S-1 reveals up to $567B compute spend and self-built data centers — jeremyakahn · 2026-10-27
- Resin lets an agent build standalone LLM inference engines; vLLM and SGLang optimize DeepSeek kernels differently — AccBalanced · 2026-10-27
- Local benchmark: Flint matches Qwen3.6 quality at 50/52 while running 3.6x faster on M4 Pro — RichardSocher · 2026-10-27
- Anthropic S-1: CSP marketplace revenue jumped from 11% to 48% in three years — AccBalanced · 2026-10-27
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27