Graph-guided LocAgent: fine-tuned Qwen-32B hits 92.7% file Acc@5 at $0.09

LocAgent: Graph-Guided LLM Agents for Code Localization

Zhaoling Chen, Xiangru Tang, Gangda Deng, Fang Wu, Jialong Wu, Zhiwei Jiang, Viktor Prasanna, Arman Cohan, Xingyao Wang

cs.SE, cs.AI, cs.CL

2025-03-12

LocAgent indexes a Python repo as a heterogeneous graph with three agent tools. Fine-tuned Qwen-32B hits 92.7% file Acc@5 at $0.09, close to Claude-3.5.

What problem this solves

Code localization is the job of pointing to the files, classes, and functions that must change given a GitHub issue. Developers already spend a large share of debugging time on that search, and automated repair fails when the search is wrong. The report often names a symptom, not the shared utility that actually needs the patch.

Two families of tools exist. Dense retrievers need a continuously updated embedding index, which lags on a moving repo. Agents such as SWE-agent and OpenHands walk directories and grep, so they miss call and inheritance edges that never appear in the issue text.

Method

LocAgent parses a Python repository into a directed heterogeneous graph. Nodes are directories, files, classes, and functions. Edges cover contain, import, invoke, and inherit. Functions are the smallest retrieval unit. Indexing is sparse: fully qualified names, a same-name dictionary, and BM25. Building the index takes a few seconds per repo.

The agent sees three tools:

Planning follows a chain of thought: extract keywords, bind them to entities, reconstruct the fault-to-failure path on the graph, then rank suspects. Reciprocal Rank across sampled iterations is used as a confidence score.

For cost, 433 successful Claude-3.5 trajectories on SWE-bench training data plus 335 successful trajectories from a first-round Qwen2.5-32B are distilled with LoRA into 7B and 32B Qwen-2.5-Coder models.

Results

SWE-Bench-Lite is filtered to 274 examples where an existing function is patched. Acc@k counts a hit only when every ground-truth location sits in the top-k list.

MethodFile Acc@5Function Acc@10
CodeRankEmbed84.67%58.76%
OpenHands + Claude-3.590.15%70.07%
LocAgent + Qwen2.5-32B (ft)92.70%77.01%
LocAgent + Claude-3.594.16%77.37%

Cost per example is about $0.66 for Claude-3.5, $0.09 for the fine-tuned 32B, and $0.05 for the 7B. With Agentless as a fixed editor, swapping in LocAgent-32B localization lifts Pass@10 from 33.58% to 36.13%; Claude localization reaches 37.59%. Pass@1 barely moves (26.31% to 26.79%).

Loc-Bench adds 560 later issues across bugs, features, security, and performance. Claude-3.5 file Acc@10 is 86.07%; the fine-tuned 7B reaches 79.64%. Bug reports remain easier than the other three categories.

Ablating SearchEntity drops function Acc@10 from 71.53% to 53.28% on the 7B setting. Capping TraverseGraph at one hop also hurts function-level scores. The extra edge types are doing real work on multi-hop cases.

Why it matters

Localization is the first hop of any repo-level coding agent. Walking a call graph instead of a directory tree lets an open 32B model sit next to Claude at roughly one-seventh the API cost. Teams on Python repos that can afford a light graph index can use this now.

The contribution is assembly, not a new primitive: graph indexes and agent tools already exist. The useful parts are a fuller relation set, more compact tool outputs, and a cheap open-model distillation.

Limitations

Python only. Fine-tuning stays on Qwen-2.5-Coder with LoRA, and the trajectories are mostly successful Claude traces, which matches the Loc-Bench drop on non-bug categories. Downstream tests use Agentless bug repair only. Acc@k is strict: missing one of several patched functions counts as failure, which is a different question from "did we find the main file."

Terms

Source

What people are saying

Related papers

All paper explainers