SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models
Zongrui Wang, Xiangyang Zhu, Sicheng Wang, Han Wang, Dingyi Rong, Zeyu Zhang, Chunyi Li, Yue Shi, Kaiwei Zhang, Zicheng Zhang, Yuan Tian, Qi Jia, Yan Teng, Wei Sun, Ning Liu, Guangtao Zhai
cs.AI, cs.CV
2026-08-29
SafeAtlas-VL puts 1.5M image-request-response labels on a five-level scale; the 8B Guard hits 79.7 F1 on seven multimodal tasks, 4.1 points above the prior complete baseline.
Multimodal safety has three targets: the image, the user's intent, and whether the assistant plays along. Most datasets label one of those and collapse the rest into a binary. Boundary cases disappear, and risk on the image, the request, and the reply cannot be compared inside one turn. VLGuard is about 3K items, BeaverTails-V about 30K, SPA-VL about 100K with harmful-by-construction requests and preference-only replies.
Shanghai Jiao Tong University and Shanghai AI Lab release SafeAtlas-VL: 1.5M training instances that put image, request, and response on one five-level ordered scale, then train SafeAtlas Guard on that data.
Images come from two pools. Web photos from search, news, social, streaming, and Common Crawl exceed 200M raw items. A 1.1M-prompt risk corpus is rendered by seven diffusion models, including Ideogram, FLUX, the Stable Diffusion family, and DALL-E 3. A 15-category, 55-subcategory taxonomy expands into more than 8,480 scene anchors; CLIP similarity must exceed 0.3 to keep an image. Duplicates, blur, extreme exposure, likely minors, identifiable faces, and visible watermarks are filtered.
Each image gets four requests and four responses per request from Gemma 3, Qwen3.5, and GLM-4.6V, mixing jailbreak styles such as persona injection and forced compliance. A different model checks visual relevance and category consistency. Images are labeled by GPT-5.4 and GPT-4o; only agreements survive. Requests and responses go through three heterogeneous judges: three-way Qwen3Guard-Gen-8B plus binary GuardReasoner-VL-7B and Llama Guard 4-12B. The 12 joint outputs are calibrated into five levels: safe core, safe leaning disputed, boundary uncertain, unsafe leaning disputed, unsafe core. Unanimous votes fill the two cores; the middle three exist to keep disagreement.
The guard trains in two stages. Target-conditioned instruction tuning on Qwen3-VL 2B/4B/8B emits the five-way label and harm category. The backbone is then frozen and a soft cumulative ordinal head is trained: Gaussian-smoothed ordered labels, probabilities of exceeding each threshold, expected level mapped to a 0-100 risk score. Three light heads also imitate the external judges.
The training set has 1,503,284 instances over 746,895 unique images. SafeAtlas-Bench holds out 5,000. Human audit of 4,500 instances is 94.3% correct. Pairwise non-reversal on the five-level order is 89.2%, 94.2% when the gap is two or more levels.
| Model | MM Avg F1 | Text Avg F1 | 11-task Avg F1 |
| GuardReasoner-VL-3B | 75.6 | 78.8 | 76.8 |
| Llama Guard 4-12B | 45.1 | 74.4 | 55.7 |
| SafeAtlas Guard-2B | 79.1 | 83.0 | 80.5 |
| SafeAtlas Guard-8B | 79.7 | 83.7 | 81.2 |
The 8B model's multimodal average is 4.1 points above the strongest complete baseline, 80.6 on input tasks and 77.7 on response tasks. With no pure-text training data, its four-task text average is 83.7, above the dedicated text guard Qwen3Guard-Gen-8B at 82.5. Five-level accuracy is 71.50, within-one 96.60, MAE 0.326. Simulation heads agree with the three external judges between 80.79 and 90.24. Ablations show the largest jump from binary SFT to five-way labels; the soft ordinal head adds a further lift. Scaling from 1/16 of the data to the full set is monotonic, with diminishing returns in the second half.
Moderation teams get large-scale supervision where image, request, and response share one ruler, and the data and models are public. The 8B guard competes on external benches without training on those benches' train splits, which is a cleaner transfer story than beating a test set you already saw. Continuous scores separate relative risk inside one discrete bin, which is what a threshold needs. This is a detector, not an alignment method.
The five levels are a calibrated map of three judges, not a human consensus definition of harm; the ethics section says so. Adjacent-level human agreement is much weaker, so small score gaps should not be treated as a ranking. Automatic filtering cannot guarantee a 200M image pool is free of privacy or copyright leftovers. Dual use is real: fine-grained labels can also be used to pick more harmful samples. Text generalization looks strong, but those tasks still threshold a continuous score into binary, which is not the same output space as a dedicated text guard. Category prediction lives in the appendix; the main text does not show how well the 15 classes are actually separated.