ICDAR 2026 Competition on Information Extraction from Atomic Layer Deposition/Etching (ALD/E) Scientific Figures
Fahad Ahmed, Sören Auer, Jennifer D'Souza
Science series (Springer Nature
cs.CV
2026-07-29
ICDAR 2026: 68 teams, 1,951 figures from 205 materials papers. Models handle classification and summarization but stall on data extraction and visual QA; the top VQA score is 0.31.
A lot of knowledge in scientific papers lives only in the figures, not the text: band diagrams, phase diagrams, spectra. Multimodal models are said to "see" images now, but do they actually understand domain-specific scientific figures? Existing benchmarks are either general images or plain document parsing (pulling text out of a PDF), neither of which tests the end-to-end skill scientists need: classify the figure, extract the plotted values, summarize, answer questions about it.
The domain chosen is a hard one: ALD/ALE (atomic layer deposition and etching), the semiconductor and materials technique that builds or removes material one atomic layer at a time. Its figures are specialized (band diagrams, phase diagrams, multi-spectra charts), far harder than general images.
The team built Sci-ImageMiner: 205 ALD/ALE papers, 1,951 figures, annotated by 10 domain experts (postgraduates and postdocs), with a 49-category figure taxonomy, split 1,180/201/570 across train/dev/test. Document parsing used MinerU, pre-annotation used Qwen2.5-VL-7B. Inter-annotator agreement was a moderate Fleiss' Kappa of 0.46.
Four tasks stack in difficulty:
The competition ran January to April 2026: 68 active teams, 1,263 submissions. Baselines were 8B-class open VLMs (Gemma, Qwen3-VL, GLM-4.6V, InternVL 3.5, Molmo2).
Top team versus best baseline per task:
| Task | Best baseline | Winner | Read |
| Classification (F1) | Gemma 0.68 | RicohSRCB 0.81 | labels: doable |
| Data extraction | Qwen3-VL 35.97 | TeleOCR-VL 41.81 | read values: low all round |
| Summarization | InternVL 0.48 | DeepVitminC 0.56 | paraphrase: near usable |
| VQA | Molmo2 0.20 | DeepVitminC 0.31 | reasoning: low ceiling |
The pattern is clear: classification and summarization (answer is a label or a paraphrase) are tractable; data extraction and VQA (read a precise value or reason over the figure) lag badly. VQA is worst, with the top team at just 0.31 weighted, and the "list" answer type near zero (0.09). No single baseline was strong everywhere.
For anyone in multimodal models or agents, this is a useful yardstick: it quantifies the gap between "summarize this figure" (works) and "what value is plotted here" (does not), which is exactly the capability scientists want. Any team building literature-reading, data-extraction, or research agents should check where it falls.
A caveat: the domain (semiconductor fabrication) is narrow, annotator agreement is only moderate, and the absolute scores are low enough that the benchmark reads more as a diagnostic than a leaderboard worth chasing.
The authors concede: moderate agreement (0.46), no baseline strong across the board, VQA baselines far below the top teams.
A sharper concern: when experts themselves only "moderately" agree, it is unclear how much of the low VQA score is a label problem versus a model problem. The "list" answer type scoring 0.09 may reflect a poorly specified task, not a model that cannot read. And 205 papers is thin for a benchmark claiming comprehensiveness, with ALD/ALE narrow enough that the findings may not transfer to, say, biology figures.