Long-Standing Bug in multiPL-e Dataset Turns Python Instructions into 'Rsthon'

A developer found a long-standing preprocessing bug in the multiPL-e dataset's MBPP portion, where a regex replacing 'py' with 'rs' turned Python instructions into 'Rsthon', exposing flaws in the benchmark's data cleaning pipeline.

2026-08-17 ~ 2026-08-17 · 2 related posts

• multiPL-e dataset contains hilarious regex errors, generating 'rsthon' tasks — code_star · 2026-08-17
• Silly regex error in multiPL-E dataset creates 'rsthon' — eliebakouch · 2026-08-17