Dev Open-Sources a 380M Data-Curation Decision Model That Filters 1M Web Pages for $5
vanstriendaniel · x · 2026-10-29
A developer trained and fully open-sourced a small specialist decision model for curating web-scale data, fine-tuned from UltraBERT-380M, complete with training data and recipe.
- Very cheap to run: processing 1M FineWeb pages took 19 minutes and $5 on Hugging Face Jobs
- The model judges topic, format, educational score, and spam; it flags a quarter of pages as product pages, only 6% pass an "educational, not spam" filter, and 1 page in 1,020,000 rated "outstanding"
- Every training-data row includes source, labeller, and licence; the author calls for more open data-prep releases to accompany open model releases
Related event: Open-Source 380M Model Curates 1M Webpages for $5(2 posts)→
More from Infra
- Ollama's two-week model drop: LFM3-1.7B targets edge agents with 256K context at ~1GB — ollama · 2026-10-30
- Meta's GenRecs Tokenization lifts ad conversions +0.35% with semantic vocabulary — Meta_Engineers · 2026-10-30
- Parallel Language Models paper: multi-token generation at AR quality, up to 5.7x speedup — aaron_lou · 2026-10-30
- Oracle investor day: 1GW AI site earns ~$80B contract value at ~28% ROI, 3.6-year payback — rohanpaul_ai · 2026-10-30
- Sam Altman: Local inference is a terrible allocation of scarce global compute — morqon · 2026-10-30
- Dual RTX 4060 Ti local LLM benchmarks: 35B MoE decodes at 122.5 tok/s — Rooneybuk · 2026-10-30