40 Days to Translate on RTX 5090? Dev Seeks Efficient Small Model
a9udn9u · reddit · 2026-08-07
A developer needs to translate 670k short English text snippets (avg 40-70 words) into 5 languages. Testing Qwen 27B and 35B models on an RTX 5090 yielded only 60-70 TPS (5 seconds per snippet), projecting a 40-day completion time.
Adjusting batch sizes did not significantly improve throughput. The author is considering smaller quantizations or MTP and asks the community for lightweight model recommendations tailored for simple, faithful product description translations.
More from Infra
- How to Run FLUX Locally on RTX 5090? Reddit User Seeks Setup Advice — RutabagaSad8257 · 2026-08-07
- AWS Veteran's Deep Dive: Building Scalable Cloud Control Planes Over 14 Years — TheNickWalsh · 2026-08-07
- Musk: Terafab Texas Will Be the Largest and Most Valuable Building on Earth — beffjezos · 2026-08-07
- VIGIL: Real-Time Terminal Dashboard for Hardware Power Monitoring — tom_doerr · 2026-08-07
- ComfyUI Node Optimizes MiniMax H3 Generation Time by 30% — nnorbbi · 2026-08-07
- DARPA Launches ScAN Program to Develop Analog Neural Networks for AI Efficiency — inductionheads · 2026-08-07