KEEP-Bench tests whether context compression preserves the evidence QA evals miss
kaiwei_chang · x · 2026-10-23
KEEP-Bench directly measures whether context compression preserves all evidence needed for reasoning. Instead of checking final answers, it decomposes references into atomic evidence propositions and judges each against the compressed context, exposing unfaithful compressions—like wrongly bound dates—that traditional QA evaluation lets pass.
More from Research
- ThinkingCap squeezes 38.5% fewer thinking tokens out of Qwen3.8-Flash-Next at ~1pp accuracy cost — Thom_Wolf · 2026-10-23
- DLOSAC refines RANSAC with Delaunay-guided spatial-geometric losses for geometry estimation — zhenjun_zhao · 2026-10-23
- New Benchmark Tests 3D Foundation Models on Geographically Diverse Architecture — zhenjun_zhao · 2026-10-23
- FineSplat: Fine-Grained Patches Push Feed-Forward 3D Gaussian Splatting to New SOTA — zhenjun_zhao · 2026-10-23
- ReDT: Resolution-Decoupled Transformer Brings 16x Speedup to HD 3D Reconstruction — zhenjun_zhao · 2026-10-23
- Causal Spatiotemporal Mamba Enables Feed-Forward Monocular 4D Reconstruction — zhenjun_zhao · 2026-10-23