pixel-curator: open-source image dataset prep with quality scoring, dedupe and leak-safe splits
Impossible-Car-2809 · reddit · 2026-10-12
A developer open-sourced pixel-curator (v0.1.0, MIT), consolidating the dataset-prep scripts they rewrote before every LoRA/Dreambooth run into one pure-Python tool.
Key features:
- Scores each image on classical-CV signals: sharpness, exposure/clipping, contrast, color cast, noise, composition — and flags weak ones
- Removes exact and near-duplicates; near-duplicates are grouped into the same train/val/test split to prevent leakage
- Exports Kohya-style folders with caption sidecars, supporting WD14 and Florence-2 captioners
- Content-agnostic core with a plugin system per domain (custom quality weights and caption vocab), ships with a worked "interiors" example
Pure NumPy/Pillow, no OpenCV, usable as CLI or library. The author is seeking feedback on the plugin API and suggestions for which verticals to support next; reminder to only use images you have rights to.
Related event: Open-Source pixel-curator Simplifies Image Dataset Preparation(2 posts)→
More from Multimodal
- From Beethoven to Suno: full breakdown of how the AI duet WANTED was made — tetsuoai · 2026-10-12
- Uncensored Qwen-Image-2.1 Quantized Hits Hugging Face Trending — abenzerps · 2026-10-12
- 4070 Ti Super beats RTX 5070 2x on MiniMax H3 int8 in local inference benchmark — smellyelon · 2026-10-12
- OpenWork, an Open-Source Claude Cowork Alternative, Hits 24k GitHub Stars — alex_verem · 2026-10-12
- Black Forest Labs confirms open-weights FLUX 3 Image is launching next — Accurate-Ad2562 · 2026-10-12
- Atlas generates unseen character views from a single image along a custom camera path — XRarchitect · 2026-10-12