40 Days to Translate on RTX 5090? Dev Seeks Efficient Small Model
a9udn9u · reddit · 2026-08-07
A developer needs to translate 670k short English text snippets (avg 40-70 words) into 5 languages. Testing Qwen 27B and 35B models on an RTX 5090 yielded only 60-70 TPS (5 seconds per snippet), projecting a 40-day completion time.
Adjusting batch sizes did not significantly improve throughput. The author is considering smaller quantizations or MTP and asks the community for lightweight model recommendations tailored for simple, faithful product description translations.
More from Infra
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
- llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16