LangChain open-sources WikiBench to measure how much codebase wikis help coding agents
LangChain · x · 2026-08-26
The LangChain team built WikiBench to evaluate OpenWiki, their open source agent for generating and maintaining codebase documentation. It answers two questions:
- How good is a given wiki? — evaluating quality differences between wikis using questions grounded in the underlying codebase.
- Does the wiki help at all? — the same questions were run three ways: wiki only, source only, and both, to see which won.
WikiBench is built on Harbor, a framework for evaluating agents on long-running tasks, whose tasks have three components. The goal is to measure whether changes actually make OpenWiki better and more useful for coding agents.
Related event: LangChain Open-Sources WikiBench to Evaluate Codebase Wiki Agents(4 posts)→
More from coding & agent
- Context layer is the backbone of production-grade agentic systems — Pavan_Belagatti · 2026-08-27
- PrimeIntellect verifiers v0.3.1: Model Interception and Persistent ACP Sessions — xeophon · 2026-08-27
- Managing shared agent context: Git, vector DB, or both? — OwlZealousideal4779 · 2026-08-27
- RAG Isn't Dead: Navigating Retrieval vs. Agentic Search — hugobowne · 2026-08-27
- Devin rebuilt its renderer for massive sessions — premqnair · 2026-08-27
- Karpathy's 1-Hour Stanford Lecture: From LLM to Prompt to Agent to Graph — AlishaOutridge · 2026-08-27