2026-09-17
A Rice LLM agent chain screened 27,830 papers, extracted 8,979 blended-cement records in under an hour, and hit F1 0.98 with Claude Sonnet 4.5.
Concrete is the most-used manufactured material and accounts for 8–9% of global anthropogenic CO2. Data-driven mix design has already cut material cost by about 30% and embodied emissions by about 60% in reported studies, but the lab data underneath those models are thin. Most machine-learning papers in this area train on fewer than 500 records, and more than 90% omit chemical or physical descriptors of the binder. The most-used public set is still Yeh's 1,030 records from 1998. Decades of experiments sit in the literature, scattered across nested tables, footnotes, and prose, with inconsistent units and acronyms. Manual extraction cannot keep up with publication volume.
Older NLP stacks such as ChemDataExtractor and MatSciBERT can pull entities, but they need parsing rules and annotated corpora, and they break when the materials system changes. Concrete is a hard case: composition is a mixture of several supplementary materials, each with its own oxides, fineness, and dosage, often split across tables.
The Rice pipeline splits a full-text XML or HTML article into a chain of specialized LLM agents, with no fine-tuning. A table agent converts tagged tables into compact plain text, then keeps only tables that hold binder properties, mix proportions, or compressive strength. Processing agents then flatten multi-level headers, fold footnotes into the table, expand acronyms, fill blank cells that inherit the row above, and map column names and units onto a shared target (lb/yd³ to kg/m³, psi to MPa). Water implied by a water-to-binder ratio, and superplasticizer given by volume or by percent of binder, get converted to mass by dedicated agents.
On the text side, regular expressions harvest candidate sentences for acronym definitions, curing temperature and humidity, and specimen geometry. An LLM then picks the relevant sentences and structures them. Mix IDs join tables and text into one record per mixture, with a sanity check that strength is positive. PDFs are avoided on purpose, because they discard logical document structure.
The same model is used for every agent in a given run. Seventeen open and closed models are scored against a human gold set of 3,015 records from 58 papers. Production extraction uses GPT-4o.
A Web of Science search found 46,820 papers after 2000; 27,830 full texts in XML or HTML were retrieved. LLM screening plus manual checks left 278 papers. GPT-4o extracted 10,313 records; after duplicate and scope filters the public database has 8,979 records, more than 100 attributes, and 3,121 unique mixtures, covering fly ash, blast-furnace slag, silica fume, limestone powder, and calcined clay. The previous largest public set had 2,171 records (Imran); Jiang's 5,026-record set was not released.
| Setting | Metric | Result |
| Claude Sonnet 4.5 | overall F1 | 0.98 |
| GPT-4o | overall F1 / precision / recall | 0.97 / 0.98 / 0.96 |
| GPT-4o, specimen size | F1 | 0.89, weakest of five categories |
| 17 models | overall F1 | all above 0.85; 12 above 0.90; 8 above 0.95 |
| human vs GPT-4o | minutes per paper | 21.6 vs 5.5 |
| 278 papers, parallel | wall clock | under 1 hour vs about 25 hours sequential |
Downstream, XGBoost predicts compressive strength. Adding binder oxides and Blaine fineness lifts in-distribution accuracy slightly, and CaO content ranks among the top features. The out-of-distribution test trains only on plain portland cement and binary blends, then predicts ternary and quaternary mixes. Even with the full training set, RMSE stays above 14 MPa. Adding target-domain samples helps: about 50 extra points reach 10 MPa RMSE with chemical descriptors, versus more than 100 without them.
This is a complete literature-to-database run for materials informatics: a modular agent chain that handles messy tables without domain fine-tuning, and that works for most of 17 models. Mix-design practitioners can train on the 8,979-record public set. People building LLM extractors get a clear failure map: tables are more reliable than prose, and specimen geometry is the category that most often picks the wrong sentence.
The contribution is assembly and scale, not a new model.
More than 60% of papers put strength in figures, and the pipeline does not read figures, so a large pool of numbers is left on the cutting-room floor. Nearly 30% of papers omit binder properties in tables, 36% omit curing conditions in the text, and 16% omit specimen size. Undefined acronyms are ambiguous: FA can mean fly ash or fine aggregate. The LLM still botches unit conversion (0.56 m²/g written as 5.6 m²/kg) and percent arithmetic (0.5% treated as 0.5×). Evaluation is end-to-end and does not isolate errors at intermediate steps. The gold set is 58 papers, about 20% of the corpus; manual extraction plus spot checks cover more than 80% of records, and the rest were not checked line by line against the source.
Only XML/HTML after 2000 is in scope. Only compressive strength is extracted, not workability, setting time, or durability, and not aggregate mineralogy. Mixes must be convertible to mass per unit volume, so pastes and mortars are often dropped.