Bytedance intern sabotage postmortem: 100+ engineers hunted the culprit for months

teortaxesTex · x · 2026-10-24

X user teortaxesTex shared a detailed postmortem (from 晚点) of the Tian Keyu case — the ex-Bytedance intern who sabotaged training of the Goku video model — commenting that he packaged vandalism as "naive justice."

Key details:

From May 2024, Goku training broke every 2-3 hours with erratic loss and unreproducible experiments; 100+ engineers spent months checking code, data, GPUs and networking, even suspecting NVIDIA GPU bugs and "solar storms";
On July 30, engineer Chen Shoufa found PyTorch data-loading code altered to randomize a previously reproducible read order; login records pointed to Tian, and the next day sabotage code was found in his hidden .temp directory, plus more traces-hiding and training-disruption programs;
Per insiders, Tian admitted multiple attacks on Chen during the internal investigation, then lost access and his internship;
Two years later he cast himself as a victim on a podcast ("only stopped the program 5 times in 10 days"), contradicting the technical evidence of auto-triggered sabotage and months of failures.

Original post →

Bytedance intern sabotage postmortem: 100+ engineers hunted the culprit for months

More from Companies & People