Jev Context Compaction: From Proposal to Failed Benchmark

Episode 1 · Jev Compaction Draws Pushback as Summarization-Style Compression Wins Favor (2026-09-18, 5 posts)

After mitsuhiko proposed using Jev for on-the-fly context compaction in agents, multiple framework and model authors publicly disputed the approach, sparking a debate over context management strategies that currently favors summarization-based compression.

Confirmed

• Theo rebutted Jev's line-by-line compaction point by point: Jev scores each tool call and drops irrelevant content in place of summarization, but compaction is not a filter.
• pvncher argued, echoing Theo, that Jev cannot access the reasoning trace behind tool calls, so it cannot understand why a tool was invoked; a homegrown compaction scheme also struggles against model-native training strategies.
• Teknium (Nous Research co-founder, Hermes author) said he repeatedly rejected related PRs for Hermes Agent; he explained the scheme deletes 250K tool-call tokens when hitting 500K context, and estimated Jev cannot even cut 50% of context—versus Hermes summarizing 95%.
• When asked to update Hermes for Jev or build a mini harness routing prompts to the cheapest model, Teknium replied Jev cannot help that much with compression.

Why it matters

• The dispute exposes a fundamental split in agent context management: scoring-based pruning versus summarization. Consensus points to pruning without understanding tool-call motivation risking critical context loss, while summarization offers far higher compression rates—a directly useful lesson for long-horizon agent builders.

• Theo tears apart Jev's 'instant compaction': line-by-line context trimming misunderstands the fundamentals — AccBalanced · 2026-09-18
• Why swapping in Jev for context compaction underperforms: models are trained on their own strategy — pvncher · 2026-09-18
• Teknium doubts Jev can deliver meaningful context compaction or cheap-model routing — Teknium · 2026-09-19
• Teknium: Jev can't compact context well — Hermes summarizes 95% of it away — Teknium · 2026-09-20
• Teknium explains why "tool-call pruning" compaction keeps getting rejected from Hermes Agent — Teknium · 2026-09-20

Episode 2 · Teknium Tests Jev Compression Scheme, Finds It Worse and Costlier (2026-09-20, 2 posts)

Teknium of NousResearch tested Tamara's Jev-based tool-call compression strategy and found it scored worse at higher cost, learning shortcuts that avoided compression entirely; he says Jev has valid use cases but this isn't one.

• Teknium Critiques Tamara's Jev Compaction Strategy: Valid Use Cases, Wrong Approach — Teknium · 2026-09-20
• Teknium benchmarks Jev compaction: 75.5% @ 115K loses to production's 78.9% @ 55K — don't adopt — max_paperclips · 2026-09-20