Walmart proposes MAREval, a multi-agent LLM framework for evaluating recommendation explanations
_reachsumit · x · 2026-10-29
Walmart researchers released the arXiv paper MAREval, a multi-agent LLM framework for evaluating natural-language recommendation explanations, addressing the cost of human evaluation and the prompt sensitivity and inconsistent outputs of single-agent approaches.
The framework has four components:
- Planner agent: uses a novel Chain of Debate (CoD) prompting strategy to coordinate roles and enforce logically consistent evaluation plans;
- Moderator agent: regulates discussions and mitigates prompt drift;
- Arbitrator agent: aggregates outputs across multiple evaluation rounds;
- Monte Carlo sampling: improves robustness and alignment with human judgment.
Experiments on the public TopicalChat benchmark and a proprietary e-commerce dataset show MAREval outperforms SOTA baselines.
More from Research
- Cross-lingual LLM effects paper wins Outstanding Paper award at EMNLP 2026 — anas_ant · 2026-10-29
- Lie detector compiled into Qwen3.5 weights flags 91% of lies with zero false positives — teortaxesTex · 2026-10-29
- Practitioner: No serious lab actually trains on the test set today — nrehiew_ · 2026-10-29
- Researchers slam secrecy around OpenAI-linked Yang-Mills proof materials — anshulkundaje · 2026-10-29
- Job-market paper: German engineering societies lifted regional patents 4.8% — Afinetheorem · 2026-10-29
- François Fleuret Skeptical of Self-Improvement Training: Only Boosts the Easy Early Part — francoisfleuret · 2026-10-29