FOVEATED Fixes Context Reliance in Unstructured Knowledge Editing Across 5 LLM Editors
Ding Wu · hf · 2026-10-06
Unstructured knowledge editing (UKE) suffers from a "context reliance" failure mode: edited LLMs can reproduce the editing passage but fail to reliably recall its individual facts without the original context. The authors trace this to difficulty underestimation under the standard passage-level objective—later facts get richer ground-truth context, yielding lower initial losses and appearing easier to learn.
They propose FOVEATED, a plug-and-play framework that builds focused views of each sentence by randomly shifting the RoPE positions of its preceding context's keys. The perturbation is applied only during editing and removed afterward, leaving inference-time positional encoding untouched.
FOVEATED is instantiated for both direct-optimization and locate-then-edit editors, with theoretical analysis and consistent empirical gains across five KE editors, two LLM backbones, and three benchmarks.
More from Research
- Hugging Face researcher: encrypted reasoning is the worst thing to happen to LLM evals — _lewtun · 2026-10-06
- WM-VLM: world model generates visual intermediate states for spatial reasoning — ZhitingHu · 2026-10-06
- ProAR turns autoregressive video models into goal-oriented prospective reasoners — muhao_chen · 2026-10-06
- 90+ Claude Opus 5.5 agents find two room-temperature magnetic semiconductor candidates in 3 days — Dr_Singularity · 2026-10-06
- Redditor Post-Trains Yandex AliceAI-80B-A3B From Scratch on 3 Local V100s, Update #5 — jjusko20 · 2026-10-06
- Double-Agent LLM Defenders: RL-Trained Theory of Mind Fools Attackers, Frontier Models Score Only 27-34% — mohitban47 · 2026-10-06