Artificial Analysis Independently Reproduces Harvey Legal Eval
ArtificialAnlys · x · 2026-07-08
Artificial Analysis released LAB-AA, an independent reproduction of the Harvey evaluation. Key differences from the original include: models run on their in-house Stirrup agent harness, which supports context compression instead of failing at the limit; it uses simplified custom agents and judge prompts; it excludes Harvey's custom tools and document generation scripts (like pptx, docx), replacing them with simple code execution tools to reflect raw model capabilities; and deliverables must strictly match specified filenames rather than using fuzzy matching.
Related event: Artificial Analysis Releases Harvey Legal Agent Benchmark Results(8 posts)→
More from coding & agent
- Astra storyboards plus Minimax H3 per-shot generation boost video success rates — Hailuo_AI · 2026-09-11
- Codex tip: use Sol with Astra and Luna sub-agents to save usage — pvncher · 2026-09-11
- agents-best-practices: a provider-neutral Agent Skill for designing and auditing agentic harnesses — tom_doerr · 2026-09-11
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- First-ever Three.js Conference lands in Paris, with a panel on AI-shortened design workflows — OdinLovis · 2026-09-11
- Agile co-author Ron Jeffries publishes 'Resist AI', urging developers to push back — mborch · 2026-09-11