River API Now Trains LLMs With Any Custom PyTorch Loss, Compiled and Run at Scale on Its GPUs
ibab · x · 2026-10-30
River AI announced that its training API now supports arbitrary custom loss functions for LLM training: you write the loss in PyTorch locally, and the River Symbolic Engine compiles and executes it efficiently across its GPU cluster.
How it works:
- Supported tensor ops are captured locally into a typed symbolic graph, validated for shapes and memory planning
- The backend codegens an optimized program; the loss and its backward pass run next to the model's logits within a single forward-backward call
- No client-side gradient exchange; memory management and parallelization are handled automatically
Why it matters: most post-training APIs force clients to compute custom losses locally, adding redundant communication and complexity. The post calls out Tinker, citing up to 1.5× the FLOPs and 3× the wall-clock time of a standard forward-backward pass.
A worked example shows decision models — general-purpose classifiers returning per-label probabilities — usable for things like agent action approval or content moderation.
More from Infra
- Shipping a $380M High-NA EUV machine: 40 containers, 3-4 Boeing 747s — Yamapama · 2026-10-31
- China could flood InP laser market next year, single vendor capacity cited at multiples of demand — jwt0625 · 2026-10-31
- AI is starting to design and build the chips it runs on, industry veterans say — pstAsiatech · 2026-10-31
- Agent Loops Are SGD Without Gradients: 234 Experiments Cut Inference to 2.32ms/step — jiayq · 2026-10-31
- Robotics stack advances on three fronts: Google Cloud's ER2 brain hits preview — lmoroney · 2026-10-31
- Community site maps the Pareto frontier for local LLMs on 2x RTX 3090 — EmilPi · 2026-10-31