AWS Adds AI Data Preparation to SageMaker: Auto-Labeling, Synthetic Data, Quality Checks

AWS ML Blog · rss · 2026-11-18

AWS announced new AI data preparation capabilities in Amazon SageMaker AI aimed at the slowest part of shipping generative AI: building accurate, well-labeled training and evaluation data.

Key capabilities: auto-labeling from unlabeled, seed-labeled or zero-label data with built-in task types (classification, NER, extractive QA, ordinal rating, summarization) and a multi-model jury whose agreement becomes per-row confidence; human review queues in SageMaker Studio built on Label Studio CE, with corrections feeding back into labeling; synthetic data generation grounded in your seed data and documents, producing reviewable code-based pipelines; data quality evaluation with business rules, built-in metrics and LLM-as-a-judge to catch duplicates, label noise and answer leakage; and data transformation via plain-language specs supporting SFT, DPO and RL formats.

Teams can call Python SDK APIs directly or equip a coding agent with the AWS AI-ML skill to orchestrate the workflow. Splunk is already using synthetic generation to scale eval datasets for its Agent Observability product. The post walks through an example workflow for building agentic-support eval data: define specs, inspect samples, run quality assessment, route ambiguous rows to experts, then iterate.

Original post →

More from coding & agent

coding & agent channel →