Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation
Guanqun Yang, Wenlong Zhang
cs.IR, cs.LG, cs.MM
2026-08-11
Randomly dropping a whole modality stream during training lifts retention 1.0 to 3.2x when a modality goes missing at serving time, at near-zero full-modality cost.
Multi-modal sequential recommendation assumes every item carries both an image and text. Real product catalogs do not: many items are missing an image or a text description. The catch is that models are trained on "all modalities present" benchmarks, so when a modality is absent at serving time, accuracy collapses. The paper notes that a vanilla multi-modal SASRec, stripped of text at test time, keeps only 40% to 73% of its full-modality HR@10.
The method is SMD (Sequential Modality Dropout), four lines of code at its core: during training, each modality stream (image, text) is independently erased with probability p for an entire user interaction history, so the whole sequence either has no image or no text. The model learns not to lean on any single modality.
The key design is that the entire history shares one modality mask, which matches reality: missingness clusters by category or source rather than appearing item by item at random. Robustness is measured as "retention," the fraction of full-modality HR@10 that survives when a modality is removed.
An optional cross-modal reconstruction loss trains the image and text projections to predict each other, adding extra backup between modalities.
Across four backbones (MM-SASRec, IISAN, MISSRec, fMRLRec) and four Amazon categories:
| Setting | Retention change |
| IISAN, text removed | 18% -> 56% (3.2x) |
| MM-SASRec, text removed (across categories) | 40-73% -> 79-97% |
| At 95% missing rate | HR@10 22% -> 61% (2.8x) |
| Simple backbone + reconstruction loss | 90% -> 98% |
In 11 of 16 backbone-by-dataset combinations, retention rose 1.0 to 3.2x at essentially no cost to full-modality accuracy.
This is a textbook four-line, architecture-agnostic, plug-and-play change aimed squarely at a real deployment problem. For multi-modal recommendation practitioners it buys robustness at near-zero cost. The retention metric is worth borrowing too: multi-modal recommenders usually report only full-modality accuracy and never test what happens when a modality goes missing.
Only image and text are covered; the method has not been extended to audio, video, or structured attributes. The cross-modal reconstruction loss is limited in value, helping only when fusion is simple and missingness is severe, and it can actually hurt on strong dynamic-fusion backbones like MISSRec. The paper handles modality absence, not corruption (image present but poor quality).