Tesla T4 ($400) runs Inkling 2 Mini at 22 tok/s for low-power agentic work

GingerTapirs · reddit · 2026-11-21

A Reddit user bought a $400 Nvidia Tesla T4 for a 10-inch home rack and ran Inkling 2 Mini with IQ3XXS quant + q40 KV cache at 128k context: 22.3 tok/s decode and 174.8 tok/s prefill. Not fast, but plenty for lightweight agentic work like document ingestion, OCR, and sorting, with low power in a half-height single-slot card. He added a $50 Taobao custom cooler with a fan that keeps it cool. Useful reference for tiny single-slot builds.

Original post →

More from Infra

Infra channel →