eBay distills GPT-4.1 relevance judge into compact multimodal Gemma-3 models for search
_reachsumit · x · 2026-10-14
eBay researchers published a paper on distilling an LLM-based relevance judgment system into efficient multimodal student models for e-commerce search relevance prediction.
- Background: query-item relevance prediction is fundamental to e-commerce search; human annotation is reliable but expensive and slow, while running large commercial LLM judges raises costs and limits throughput
- Pipeline: curate a small high-quality human-annotated dataset as ground truth, build a reliable LLM-as-a-judge pipeline via careful prompt engineering and iterative refinement, then use it to generate large-scale labels for training compact multimodal student models (Gemma-3)
- Results: distilled models come close to the teacher while significantly improving latency, cost, and scalability, enabling practical deployment
A representative case of LLM distillation deployed in industrial search.
More from Research
- Weaviate 1.40 ships PathSeer, a dynamic filtered-ANN algorithm fastest in nearly all benchmarks — CShorten30 · 2026-10-14
- Open Co-Scientist: open-source recreation of Google's AI Co-Scientist with Elo-ranked hypotheses — Haunting-Stretch8069 · 2026-10-14
- Mining Qwen reasoning traces to find the exact prompt phrases that derail your agent — oleg_ivye · 2026-10-14
- AIPOCH's Open-Science ships portable .science packages that share full research context, not just outputs — ahuja_priyank · 2026-10-14
- Video-Arena: Game-Based Video Benchmark Shows Native Models Score Just 13.2 F1 — Kuan Zhang · 2026-10-14
- LSCBench: Vibe Coding Agents Write Functional Code, But Only 25% Is Also Secure — Qwen · 2026-10-14