The Last Translation Benchmark Initiative
zouharvi · x · 2026-07-17
The "Last Translation Benchmark" initiative involves:
- Collecting hard-to-translate inputs covering text, images, and audio
- Requiring attached human-readable validation rules
The goal is to make future translation task evaluations more provable and stable. The author included several examples showcasing the real-world difficult scenarios it aims to capture.
Related event: Last Translation Benchmark Seeks Hard-to-Translate Samples(7 posts)→
More from Research
- Fixing compute convention and fitted floor cuts scaling forecast error ~79% — dlwh · 2026-10-27
- Retrospective: Hero scaling forecast fixed without using Hero losses, code open-sourced — dlwh · 2026-10-27
- Scaling-fit discrepancy: output-head FLOPs are ~65% of compute at the smallest rung, ~3% at 535B — dlwh · 2026-10-27
- Marin's dlwh Explains the Scaling Law Fit: Estimating the Loss Floor Early — dlwh · 2026-10-27
- Marin's Open Scaling Law Extrapolates 300× and Predicts Loss Within 0.2% — dlwh · 2026-10-27
- Delegating scaling-law asymptote fitting to Codex while planning a birthday party — dlwh · 2026-10-27