Data poisoning research suggests user fragments could shape 'AI discoveries' like Navier–Stokes
MaxDev0 · reddit · 2026-09-09
The dispute around OpenAI's Navier–Stokes result and mathematicians Tristan Buckmaster and Levent Alpöge raises an under-discussed question: how much do millions of users' half-finished ideas contribute to what get called 'AI discoveries'?
The author cites data-poisoning research from the UK AI Security Institute, Anthropic and others: 250 poisoned documents (0.00016% of tokens) reliably implanted a backdoor in a 13B model trained on 260B tokens, undiluted by clean data scale; 50-90 poisoned examples gave 80%+ attack success on GPT-3.5 fine-tuning. This undermines the 'dilution' argument—small amounts of consistent, targeted data have outsized effects, meaning researcher fragments shared with LLMs could aggregate into frontier labs' training pipelines.
More from AGI Musings
- Scooped Researchers Publish Less in Top Journals and Get 21% Fewer Citations — soumitrashukla9 · 2026-09-09
- Newton with 150 Hires? OpenAI Scooping Row Draws Leibniz Analogy — JMannhart · 2026-09-09
- Dev welcomes labs training on his code: better models benefit me too — intellectronica · 2026-09-09
- Tom Dietterich: AI safety hinges on novel, ill-defined tasks AI can't optimize — tdietterich · 2026-09-09
- Tom Dietterich clarifies: 'new knowledge' in AI debate means environment-interaction knowledge — tdietterich · 2026-09-09
- From GPT-4 failing basic addition to 10,000 agents solving a Millennium Prize problem in 3.5 years — MartinSignoux · 2026-09-09