Netflix reshapes artwork and video preview personalization with CLIP and in-house tri-modal MediaFM
_reachsumit · x · 2026-08-20
A Netflix paper details how multimodal embeddings reshaped its production recommender systems:
- Artwork personalization: augmenting a two-tower model with CLIP image embeddings lets a single model serve all five Netflix artwork canvas types, replacing five separately trained per-canvas models and substantially improving cold-start; a lightweight extension reuses CLIP's joint text-image space for query-aware artwork ranking in search.
- Video preview personalization: the in-house tri-modal foundation model MediaFM fuses visual (SeqCLIP), audio (wav2vec 2.0), and timed-text signals trained on shots from the Netflix catalog, outperforming strong visual-only baselines offline and in online A/B tests.
- The paper also shares transferable lessons for adopting foundation-model embeddings in recommenders, including a simple offline proxy task for pre-launch evaluation.
More from Research
- TMLR Survey: The Embodiment Gap in Robot Foundation Models — HirokatuKataoka · 2026-08-20
- Open Source Tool: Turn Documents into Knowledge Graphs via CLI — tom_doerr · 2026-08-20
- IIT Bombay finds the most manipulative prompt was also the most polite one — alex_verem · 2026-08-20
- Scholars Critique scFM Review: Definitions Too Loose, Mixes General and Specialized Models — jmuiuc · 2026-08-20
- KAIST STAI group opens applications for Autumn 2027 PhD programs — coallaoh · 2026-08-20
- Liquid AI's QAD technique lets Q4 models retain 97% of BF16 performance — alexcovo_eth · 2026-08-20