Overfiltering Wastes Compute; Data Selection Beats Simple Heuristics
giffmana · x · 2026-08-22
Arguing against naive data filtering, the author distinguishes between filtering and selection. Simple heuristics (e.g., filename rules) remove garbage but also discard valuable data (e.g., date-named images), wasting compute. In scenarios of limited resources or non-general models, data selection is preferred over crude filtering. The quoted researcher adds that the value lies more in importance weighting and selecting source data for rephrasing rather than precise filters.
Related event: Filtering vs. Selection: Debating Training Data Preprocessing Strategies(5 posts)→
More from Research
- U of T team mines 122,194 patient reviews across 45 Ontario hospitals with NLP to expose satisfaction and equity patterns — ishtiaqueSIA · 2026-10-16
- University of Toronto hiring AI postdoc: builders of interventions, not just benchmarkers, due Oct 30 — ishtiaqueSIA · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Mech Interp Veteran Rethinks Alignment, Misgeneralization, CoT Oversight — akbirthko · 2026-10-16
- WEDGE paper generates training environments conditioned on navigation agent failures — _akhaliq · 2026-10-16
- RExBench: best coding agent completes only 33% of autonomous research extensions — najoungkim · 2026-10-16