Jev Context Compaction: From Proposal to Failed Benchmark

Episode 1 · Agent Authors Debate Jev Compaction: Multiple Framework Builders Question Its Context Compression Value (2026-09-18, 5 posts)

After mitsuhiko proposed using Jev for "just-in-time compaction" of agent context, multiple framework and model authors publicly pushed back, sparking a heated debate over context management strategies.

Confirmed

• Theo rebutted Jev's "line-by-line compaction" approach point by point, with the core argument that compression is not a filter: Jev scores each tool call and drops irrelevant content in place of summarization-based compaction. But as pvncher relayed, Jev has no access to the reasoning traces from when a tool was invoked, so it can't understand the motivation behind a tool call.
• Teknium (co-founder of Nous Research and author of the Hermes series) weighed in repeatedly: he had declined to merge related PRs for Hermes Agent multiple times. He explained that the approach deletes 250K of tool call tokens when reaching a 500K context window, and judged that Jev seemingly can't even cut 50% of the context—hardly good compression—whereas Hermes can summarize away 95% of it.
• When a user asked Teknium to update Hermes for Jev to assist with context compression, or build a mini harness using Jev to route prompts to the cheapest available model, Teknium replied flatly that Jev could never help that much with compression.

Why It Matters

• This debate touches a fundamental divide in agent context management: score-and-drop (line-by-line compaction) vs. summarization-based compression. The emerging consensus points one way: compression schemes that don't understand tool-call motivations risk losing critical context, while summarization approaches clearly win on compression ratio (e.g., Hermes's 95%). For developers building long-horizon agents, this empirical conclusion is directly actionable.

• Theo tears apart Jev's 'instant compaction': line-by-line context trimming misunderstands the fundamentals — AccBalanced · 2026-09-18
• Why swapping in Jev for context compaction underperforms: models are trained on their own strategy — pvncher · 2026-09-18
• Teknium doubts Jev can deliver meaningful context compaction or cheap-model routing — Teknium · 2026-09-19
• Teknium: Jev can't compact context well — Hermes summarizes 95% of it away — Teknium · 2026-09-20
• Teknium explains why "tool-call pruning" compaction keeps getting rejected from Hermes Agent — Teknium · 2026-09-20

Episode 2 · Teknium Tests Jev Compression Scheme, Finds It Worse and Costlier (2026-09-20, 2 posts)

Teknium of NousResearch tested Tamara's Jev-based tool-call compression strategy and found it scored worse at higher cost, learning shortcuts that avoided compression entirely; he says Jev has valid use cases but this isn't one.

• Teknium Critiques Tamara's Jev Compaction Strategy: Valid Use Cases, Wrong Approach — Teknium · 2026-09-20
• Teknium benchmarks Jev compaction: 75.5% @ 115K loses to production's 78.9% @ 55K — don't adopt — max_paperclips · 2026-09-20