15 self-funded GPU jobs show cost estimators overshoot by a third

Worldly_North_7213 · reddit · 2026-09-14

The author spent $33.60 of their own money running 15 jobs across three GPU providers (7B QLoRA, bf16 LoRA, bge-large embeddings, image jobs), froze an estimator built from published specs, then compared quotes to actual bills.

Results:

First 9 runs: mean error 33%, always over. Causes: overestimated provisioning (assumed 10-14 min setup; containers live in 5-10s) and stale throughput priors (A6000 1.8x faster than spec, L40S 1.4x, A100 1.3x).
After recalibrating on measured throughput: unseen-config runs within 20% (3/6), but extrapolation fails badly: +85% on an unmeasured precision setting, +151% on a CPU-bound job, +46% on a spot run with three restarts.

Key numbers (same 7B fine-tune, H100-SXM at $3.29/h): QLoRA nf4 cost $3.14/56 min; bf16 LoRA $1.50/26.5 min using 71/80GB VRAM (gradient checkpointing differed between runs). On 80GB cards nf4 was the most expensive default; on 24GB it's the only option (H100 lacks native FP4; Blackwell may differ).

Per million training tokens: $0.20 (spot 4090), $0.30 (H100 bf16), $0.63 (H100 QLoRA) vs managed fine-tuning list prices of $0.48-0.50.

Takeaway: one measured run per config beats any spreadsheet — recalibrated re-quotes landed within 8% (in-sample fit).

Original post →

15 self-funded GPU jobs show cost estimators overshoot by a third

More from Infra