Open-Source Image Dataset Prep Tool pixel-curator: Scoring, Dedup, Leak Prevention
Impossible-Car-2809 · reddit · 2026-10-12
A developer has open-sourced pixel-curator, a dataset preparation tool for LoRA/Dreambooth training (MIT, pure Python):
- Quality scoring: Scores each image on classic CV signals — sharpness, exposure/overexposure, contrast, color cast, noise, composition — and auto-flags weak images.
- Dedup and leak prevention: Removes exact and near-duplicate images, and near-duplicates are assigned to the same train/val/test split to prevent cross-set leakage.
- Export: Generates Kohya-style directories and caption sidecar files, with support for WD14 and Florence-2 tagging.
- Extensible: The core is content-agnostic; each domain is a small plugin with custom quality weights and caption vocabularies (an interiors example is built in); implemented in pure NumPy/Pillow with no OpenCV dependency, usable via CLI or as a library.
Currently at an early 0.1.0 release; the author is seeking feedback on the plugin API and suggestions for verticals worth supporting.
Related event: Open-Source pixel-curator Simplifies Image Dataset Preparation(2 posts)→
More from coding & agent
- Screego: open-source self-hosted screen sharing for devs with 10.5k GitHub stars — thisguyknowsai · 2026-10-12
- Grok app adds Bots: users can now build custom bots like Trader inside Grok — nima_owji · 2026-10-12
- Moli: open-source headless browser for AI agents renders pixels only on demand — abhishek__AI · 2026-10-12
- Open-source real-time object detection with YOLOv8, OpenCV, and threaded frame buffering — mariamamhan · 2026-10-12
- Claude Opus 5.5 makes a Blender short film about the last black hole — 3,000 lines of Python, no gen models — toni_btrain · 2026-10-12
- HF Engineer Runs pi-web on DGX Spark to Steer Local AI Agents From His Phone — lhoestq · 2026-10-12