Tsinghua's Jev-LDE boosts ICL accuracy from 81.2% to 88.1% with a single demo edit
机器之心 · wechat · 2026-10-14
Researchers from Tsinghua's Automation Department propose Local Demonstration Editing (LDE) and a small editor model, Jev-LDE, that applies one Keep/Delete/Replace edit to a retrieved demonstration set before LLM inference, avoiding combinatorial search and repeated target-model calls.
Key results
- Beats Semantic TopK init in 46 of 48 classification configs; average one-shot accuracy rises from 81.2% to 88.1%
- Trained with rewards only from Meta-Llama-3-8B-Instruct, yet transfers to Qwen targets, DeepSeek-V4-Flash, and held-out benchmarks
- Works on top of BM25, CASE, and TopK+ConE selectors in all 24 tested combinations
Method
- Editor is a Qwen3-1.7B emitting structured JSON actions, trained via GRPO with rewards from whether the frozen target LLM answers correctly
- End-to-end wall time grows only 11% (82.1s → 91.1s), far cheaper than scoring-heavy methods like TopK+ConE
- Ablations show random edits yield nothing; the RL-trained policy is what matters
Limits: gains are concentrated on classification; math reasoning (GSM8K, AQuA-RAT) shows no benefit or slight drops, and a single edit can't fix candidates outside the pool or multi-slot errors.
More from Research
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23