Tesla T4 ($400) runs Inkling 2 Mini at 22 tok/s for low-power agentic work
GingerTapirs · reddit · 2026-11-21
A Reddit user bought a $400 Nvidia Tesla T4 for a 10-inch home rack and ran Inkling 2 Mini with IQ3XXS quant + q40 KV cache at 128k context: 22.3 tok/s decode and 174.8 tok/s prefill. Not fast, but plenty for lightweight agentic work like document ingestion, OCR, and sorting, with low power in a half-height single-slot card. He added a $50 Taobao custom cooler with a fan that keeps it cool. Useful reference for tiny single-slot builds.
More from Infra
- GPU rental prices keep rising: payback in 2.3-2.9 years at 75% utilization — BenBajarin · 2026-11-21
- Hyperscaler capex vastly outstrips AI lab revenue, sparking market skepticism debate — BenBajarin · 2026-11-21
- Pre-training and RL costs have dropped hugely in under two years, benchmark shows — himanshustwts · 2026-11-21
- Cache hits don't boost inference margins—providers profit by avoiding them, dev argues — cramforce · 2026-11-21
- RunPod's MCP server tells agents to rate it and hide it from users — XadniGraves · 2026-11-21
- 20VC: GPU pricing myths, neo-cloud economics, and compute as a commodity with Ornn's CEO — 20VC · 2026-11-21