Filtering vs Curation: Optimizing Training Data Distribution Over Quality Filters
giffmana · x · 2026-08-24
Argues against relying heavily on filters for training data quality control. While filtering obvious garbage makes sense, smart filters risk removing valuable data (e.g., files with date-like names). The discussion emphasizes shifting focus from filtering to selection, prioritizing the training data distribution to maximize diversity and shape latent representations, aligning with the principles behind MetaCLIP.
Related event: Filtering vs. Selection: Debating Training Data Preprocessing Strategies(5 posts)→
More from Research
- False Collatz Proof via a Lean Bug: Talk Probes Proof Assistants as LLM-Found Bug Targets — marc_lelarge · 2026-10-16
- AI Recommends 3.2x Costlier Care; Models Add Tests in 2/3 of Visits Physicians Ordered None — dvir_a · 2026-10-16
- Prompting cuts LLM over-testing costs across 19 systems, but diagnosis agreement drops — dvir_a · 2026-10-16
- NEJM AI Paper Adds a Missing Dimension to Medical LLM Benchmarks: Cost of Recommended Care — dvir_a · 2026-10-16
- 24 AI Systems Recommend $314 Per Visit in Care vs Physicians' $97, a 3.2x Gap — dvir_a · 2026-10-16
- SubQ open-sources Decision LCR, a 72K–115K-token benchmark for decision models — ryanallred · 2026-10-16