Quantifying Contextual Interference with Two Metrics
sanmikoyejo · x · 2026-07-16
The author observes that this context-induced instability exists across multiple SOTA models and proposes two complementary metrics to quantify it:
- INS: Measures the average sensitivity per sample
- WTD: Measures the performance drop in the most severely affected samples
This explains how they transform "contextual interference" from a phenomenon into a measurable metric.
More from Research
- New survey maps how agentic systems are learning to improve themselves — SchmidhuberAI · 2026-07-21
- Jacob Tsimerman interview frames LLMs as a turning point for mathematical discovery — stevenstrogatz · 2026-07-21
- New survey bridges continual learning and parameter-efficient fine-tuning — v_lomonaco · 2026-07-21
- Daniel Hanchen’s 2-hour workshop covers open models, reward hacking and RL — danielhanchen · 2026-07-21
- Codex’s claimed proof of a math problem turns into a “CEO of math” meme — builderjaydub · 2026-07-21
- Tau Ceti launches as an AI-formalized mathematics library for Lean — wellecks · 2026-07-21