2026-09-14
I3LUNG's 9-feature clinical-blood model hit AUC 0.74-0.77 on a 2396-patient NSCLC immunotherapy test set, beating PD-L1; multimodal CV gains failed on test and external validation.
Immunotherapy gives long-term benefit to about 20–30% of patients with advanced non-small cell lung cancer (NSCLC). First-line triage still leans on PD-L1 and a handful of clinical scores. PD-L1 is the only approved predictive marker, and it is a weak one. Most patients develop primary (5–20%) or secondary (60–85%) resistance. The cost of a wrong call is toxicity, spend, and missed intensification.
Earlier multimodal models already fused CT, pathology, and genomics to predict immunotherapy benefit. Cohorts sat around 250–300 patients, with roughly 80 complete multimodal cases, and they lacked a held-out test set plus a geographic external check. I3LUNG is a Horizon Europe multicenter study (NCT05537922). This paper is the retrospective arm: six centers, stage IIIC–IVB patients treated with immunotherapy or chemo-immunotherapy from September 2012 to October 2023, 2,396 people. The goal is a physician-facing predictor, not another single-center AUC.
The main analysis uses C23, first-line metastatic (C2, n=1,437) plus later-line (C3, n=638) patients, 2,075 in total. The University of Chicago is held out as EXVAL (n=251). The remaining 1,824 are split 85/15 into TRAIN (n=1,550) and TEST (n=274). Cross-validation is leave-one-center-out, five folds. Only 339 patients have all four modalities (clinical-blood, CT, pathology, genomics); 400 have clinical-blood plus both image types.
Clinical-blood (CB) is reduced to nine features: sex, ECOG performance status, smoking, PD-L1, bone/liver/brain metastases, neutrophil-to-lymphocyte ratio (NLR), and lactate dehydrogenase (LDH). Imaging has two tracks: handcrafted PyRadiomics and foundation-model CT embeddings. Pathology slides go through Prov-GigaPath, which pools tumor tiles into one slide vector. Two model families:
Primary endpoints: overall survival at 24 months (OS24, long benefit), at 6 months (OS6, very poor prognosis), disease control rate (DCR: CR/PR/SD vs PD), and C-index on overall survival. The usability study uses only CB-only logistic regression (DCR) and Cox (OS) with SHAP. Multimodal models never reach the physicians.
C23 CB-only machine learning holds up on TEST and drops on EXVAL:
| Endpoint | TEST AUC | EXVAL AUC |
| OS24 | 0.74 (n=226) | 0.60 (n=190) |
| OS6 | 0.69 (n=259) | 0.64 (n=240) |
| DCR | 0.71 (n=273) | 0.55 (n=251) |
Deep CB-only is similar: OS24 0.73 TEST / 0.63 EXVAL, OS6 0.72 on both, DCR 0.70 / 0.65. The discussion cites TEST AUC up to 0.77 in the C2 subset. On TEST, CB-only beats single markers for OS24: PD-L1 0.53, ECOG 0.62, LDH 0.58, NLR 0.66. Survival stratification also beats the clinical LIPI score: Cox-ML C-index 0.65 versus LIPI 0.55. In a TEST subset with LDH and NLR (n=141), machine learning separates low vs high risk at 25.1 vs 4.7 months median survival.
Multimodal gains live in cross-validation. On C23 CV, CB+pathology+PyRadiomics reaches OS24 AUC 0.72 against 0.60 CB-only on the same patients (P=0.0158). Swapping in foundation-model radiology features lifts DCR to 0.77 vs 0.63 (P=0.0014). The IO-only OS24 subgroup hits 0.91 vs 0.54. Those lifts do not hold stably on TEST or EXVAL. DLIF plus images and genomics never beats CB-only on any endpoint.
Twenty physicians (10 lung experts, 10 non-experts) each read 10 real TEST cases, 200 assessments, first blind and then with SHAP. DCR sensitivity rose from 0.72 to 0.87 (P=0.0011) and accuracy from 0.57 to 0.65 (P=0.043), with a small specificity drop. Experts and non-experts both moved. The overall odds of a correct DCR call were 1.37, P=0.1, not significant. OS-range scoring improved 36% across all physicians, per the discussion. Fairness: the Greek MH center had OS6 TPR 0.96; on DCR, MH 0.96 vs Italian INT 0.63 (P=0.037). Sex differences were not significant. SHAP repeatedly flags ECOG>0 and bone/liver/brain metastases as negative for OS24; NLR matters for OS24, high LDH pushes DCR toward non-benefit.
For medical AI, this is a real-world stress test of "multimodal is always better." 2,396 patients is large; 339 complete four-modality cases is not. What beats PD-L1 on a held-out test set is nine chart-and-lab items every clinic already collects, not a CT-plus-pathology splice. After SHAP, physicians miss fewer patients who might benefit. That is closer to usability than another 0.05 of cross-validation AUC.
EXVAL median overall survival is 22.2 months against 12.4 in TRAIN. Case mix and treatment patterns differ. The drop to 0.55–0.72 reads as distribution shift, not a model that only memorized the training centers. A prospective, calibration-aware study in more than 2,000 patients is already running. This paper is not a deployment result.
The authors are direct: retrospective messy data; too few complete multimodal cases, which hurts DLIF; MLEF cannot ingest missing modalities, so clinical use needs a more forgiving architecture; genomics is sparse and radiomics covers only the primary lesion; about 15% of CTs are not pre-immunotherapy baselines; EXVAL is a single center. Subgroup P values are unadjusted and hypothesis-generating.
The usability study is not a prospective decision-impact trial. Physicians scored historical TEST cases on DCR and survival bins. Nobody measured a change in treatment or in patient outcome. The sensitivity gain on DCR is real; the overall correctness claim is P=0.1. On non-IO cohorts, response AUCs fall to 0.50–0.58, which the authors read as IO-specific signal. Those controls are smaller and asymmetrically processed, so that claim stays a hypothesis. Center-level sensitivity gaps are large, and ECOG scoring itself varies by site, so the model is not the only suspect.