LangChain open-sources WikiBench to measure how much codebase wikis help coding agents
LangChain · x · 2026-08-26
The LangChain team built WikiBench to evaluate OpenWiki, their open source agent for generating and maintaining codebase documentation. It answers two questions:
- How good is a given wiki? — evaluating quality differences between wikis using questions grounded in the underlying codebase.
- Does the wiki help at all? — the same questions were run three ways: wiki only, source only, and both, to see which won.
WikiBench is built on Harbor, a framework for evaluating agents on long-running tasks, whose tasks have three components. The goal is to measure whether changes actually make OpenWiki better and more useful for coding agents.
Related event: LangChain Open-Sources WikiBench to Evaluate Codebase Wiki Agents(4 posts)→
More from coding & agent
- Dev bills $20,000 for an app built with $200 of Claude Code — IgorCarron · 2026-10-15
- Dan Shipper: run product and labs in parallel, even with a labs team of one — danshipper · 2026-10-15
- Gemini 4 Argon wrote 5,000+ lines of pure Python to build its own ray-tracing 3D engine — aitrendz_xyz · 2026-10-15
- Minecraft fully ported from Java to Rust, file by file, server and client included — Vjeux · 2026-10-15
- mitsuhiko builds a full Python IDE for iOS in an afternoon with new AI models — mitsuhiko · 2026-10-15
- How to get Gemini 4 Argon to build games that actually look decent — chongdashu · 2026-10-15