Open Source Dataset Curation Tool Cull Updated: Local Scraping with Multimodal Filtering
Compunerd3 · reddit · 2026-08-23
The author released an update for Cull, an open-source dataset curation tool that runs locally. It supports scraping images and videos from platforms like Civitai, X, Reddit, and Discord, integrating vision models (LM Studio, Ollama, Groq, OpenAI) for automated captioning and classification.
Key features include:
- Smart Filtering: Filters based on quality scores, topic relevance, and whitelisted/blacklisted terms.
- Deduplication & Watermark Detection: Deduplicates at the source and identifies watermarks for separate review.
- Auto-Captioning: Generates SD Prompts, Booru Tags, or natural language descriptions for content lacking metadata.
- Parallel Processing & Export: Manages parallel tasks with a shared compute queue and exports to local datasets or Hugging Face.
The tool uses plain disk files (no database), is MIT-licensed, and offers a one-line Docker setup.
More from Apps
- Can ChatGPT Actually Run a Household: Scheduling, Wedding Planning and Meal Prep? — Wrong_Examination944 · 2026-08-23
- AI agent avatars criticized as nightmare fuel — jasonkneen · 2026-08-23
- Life with an AI Agent: How Bob Integrated into a Family Home — dralandthompson · 2026-08-23
- Google releases Gemma Translator, an offline voice translator running on-device with LiteRT-LM — tom_doerr · 2026-08-23
- Adapter Tool Claims to Reduce Token Usage by 90% Compared to RAG — Prof_DavidBader · 2026-08-23
- First personalized mRNA cancer vaccine wins Phase 3 trial — rand_longevity · 2026-08-23