Inference startup: training wins inference workloads, models should improve with use
ypatil125 · x · 2026-09-23
An inference provider shares months of customer-facing lessons: the business took off, but training wins and keeps inference workloads. Inference isn't a commodity when you can improve the model itself — using online training like On-Policy Self Distillation and frontier-grade RL post-training to target specific behaviors against customer evals. The thesis: models should get better the more you use them.
More from Infra
- AWS Open-Sources Strands Harness, Claims 28% Fewer Agent Tokens — shashib · 2026-09-23
- DIY multi-GPU cooling: case airflow tuning drops temps from 80C+ to 68C, no liquid cooling needed — HankYeomans · 2026-09-23
- Alibaba accelerates global AI push with new data centers across Europe and the Middle East — Polymarket · 2026-09-23
- You run kernels, not models: why the same model and GPU can perform wildly differently — Roger_M_Taylor · 2026-09-23
- Apple's Mac mini and Mac Studio get major AI-focused performance leap — BLUECOW009 · 2026-09-23
- Alibaba's V900 chip rivals Blackwell on spec, but holds just 16% China share — pstAsiatech · 2026-09-23