File-system agents beat DocLens by 4.3 points and run up to 10.3× faster

DocFS: File Systems as Agentic Reasoning Environments for Complex Multimodal Document Question Answering

Bingdian Zhang, Zixuan Li, Long Bai, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng

cs.IR

2026-11-07

DocFS stores long documents as a file tree with cross-modal links. With Qwen3.5-Plus it beats the best training-free baseline by 2.9 and 4.3 points, at up to 10.3× the speed.

What problem this solves

Complex multimodal document QA stalls on scattered evidence. A file runs to tens or hundreds of pages. Text, figures, tables, and equations sit inside a section tree, and the prose points at visuals that live somewhere else. To answer, an agent has to pick a section, open the right modality, and check the source page in case parsing dropped a detail.

Flat methods squash the file into page images or chunks, then retrieve or page through them. Hierarchy and cross-modal links never become a path the agent can walk. Content trees, multi-granularity indexes, and page graphs do write the structure down, but they mostly serve retrieval. The agent receives passages, and it still has no general operations lined up with that structure.

A file system already has both pieces. Nested directories are the sections. Typed files hold text and visuals. ls, grep, and read need no new API. DocFS turns the raw document into that environment. The work is from the Institute of Computing Technology, Chinese Academy of Sciences.

Method

Parsing is repaired before any directory is built. A parser extracts text, titles, figures, tables, and equations page by page, with page numbers and boxes. Titles from different levels often collapse into one, body text is sometimes marked as a title, and equations and tables are misread. Titles plus their context go to an LLM, which rebuilds the hierarchy and sends false titles back to body text. Cropped equation and table regions go to a VLM, a model that reads images and text together, which checks the LaTeX and the HTML against the pixels.

The repaired outline becomes a directory tree. Prose is Markdown. Figures are JPGs. Tables are kept as an image plus CSV and HTML. The root and every section get an index file and a relation file. The index lists sections and attachments at that level and is the way in. The relation file records links between sections, between a section and its figures or tables, and across modalities, so one hit can be expanded to the neighboring evidence. Each image also gets a text description mapped back to the original file, which is what lets keyword search find a figure. The description is only an entry point. Small objects and exact numbers still have to be read off the crop. The source PDF stays, with page pointers, for the last check.

The agent has four ordinary file tools. readfile opens text, an image, or a page range of the source PDF. grep searches by keyword and returns the hit plus its path. ls lists the current directory. glob collects files by pattern.

Reasoning follows those tools. The agent reads the directory, marks sections that fit the question, and runs a global keyword search to get candidate paths. It then opens those files and walks relation links to fill in other sections and modalities. Last, it looks at the cropped image and the matching PDF pages, and answers from that fine detail together with the text already gathered.

Results

The test sets are MMLongBench-Doc and FinRAGBench-V. The first covers long documents across domains, includes unanswerable questions, and uses rule-based scoring. The second has denser layouts and is scored by another model acting as a judge. Qwen3.5-Plus runs with thinking on, so it writes an internal trace before the answer. DeepSeek-V4.1-Flash runs with thinking off. On a matched backbone, the strongest baseline the paper reports is a reproduction of DocLens.

MethodMetricResult
DocFS + Qwen3.5-PlusMMLongBench-Doc accuracy / F171.6% / 70.0%
DocLens + Qwen3.5-Plussame68.7% / 66.1%
DocFS + DeepSeek-V4.1-Flashsame69.7% / 67.5%
DocLens + DeepSeek-V4.1-Flashsame67.8% / 66.1%
DocFS, both backbonesFinRAGBench-V accuracy82.3% each
DocLens + Qwen / DeepSeekFinRAGBench-V accuracy78.0% / 74.4%

Against DocLens on the same backbone, Qwen gains 2.9 accuracy points and 3.9 F1 points on MMLongBench-Doc, and 4.3 points on FinRAGBench-V. DeepSeek gains 1.9, 1.4, and 7.9 points. The gains sit mostly on charts and text. With Qwen on MMLongBench-Doc, chart is up 9.3, text 6.9, figure 2.9, and table 2.1. On FinRAGBench-V, chart is up 12.1 and 6.3 for the two backbones, and DeepSeek's text score is up 15.9. Qwen also slips: layout falls from 71.2 to 68.4, unanswerable questions from 75.4 to 73.7, and FinRAGBench-V tables from 83.3 to 79.9.

The ablations are uneven. Removing grep drops Qwen on MMLongBench-Doc from 71.6 to 62.0, the largest cut. On that same setup, removing the file system, preprocessing, the index and relation files, image descriptions, or ls lands between 66.3 and 66.9. On FinRAGBench-V, removing grep drops Qwen from 82.3 to 74.0. On MMLongBench-Doc, DeepSeek falls from 69.7 to 58.7 without preprocessing, an 11.0 point drop, and to 56.6 without grep. Without image descriptions, Qwen's chart and figure subsets fall 11.3 and 7.3 points, and DeepSeek's fall 15.8 and 10.8.

The latency and token numbers below cover reasoning only. Parsing and file-system construction run offline, and their wall clock depends on how parallel the tools are, so they stay out of the table.

SettingDocLens time / tokensDocFS time / tokensSpeedup / token cut
Qwen, MMLongBench-Doc359.9s / 217.9K43.9s / 142.9K8.2× / 1.5×
DeepSeek, MMLongBench-Doc76.8s / 407.0K17.8s / 180.9K4.3× / 2.3×
Qwen, FinRAGBench-V436.1s / 602.4K52.7s / 185.0K8.3× / 3.3×
DeepSeek, FinRAGBench-V189.9s / 1555.9K18.4s / 154.7K10.3× / 10.1×

DocFS stays around 140K to 190K reasoning tokens on both benchmarks: 142.9K and 185.0K for Qwen, 180.9K and 154.7K for DeepSeek. DocLens rises 2.8× on Qwen and 3.8× on DeepSeek once the layout gets denser. readfile is called most, grep is second, and glob is almost never used. ls is used more on FinRAGBench-V. Structure files and prose account for about half of reads, the source PDF for about a fifth, and image descriptions are read least.

One trace lines up a color-coded architecture figure in section 9 with a numeric table in section 10. Reading the figure maps blue to Conscientiousness. Another misses at first: the balance sheet has no 2019 column, a second search for 2019 reaches the stockholders' equity table, and the agent checks PDF pages 75 and 77.

Why it matters

What transfers is the environment. Directories, index files, relation files, image descriptions, and a source-file check all sit on tools agents already know how to call. Accuracy gains run from 1.9 to 7.9 points, nearer the low end on the long-document benchmark and the high end on the finance one. The larger gap is reasoning cost, and it grows as layouts get denser. Layout questions and some table questions do not improve with the rest.

Limitations

Three items are left open. The organization still has to change when document types and questions change. Cross-modal links need stronger navigation. There is no study of how to trade extra exploration against answer reliability.

Comparisons with MACT, MDocAgent, SimpleDoc, and the RAG systems use each paper's reported peak, on backbones from smaller open VLMs to GPT-4o and Gemini-2.5-Pro. Those are not matched conditions. The comparison that holds is the reproduced DocLens. Qwen thinks and DeepSeek does not, yet both score 82.3% on FinRAGBench-V. That tie does not show the method closed the model gap. The finance benchmark is graded by a model, and judge error is not reported on its own.

The efficiency table leaves out construction. Adding document-processing tokens, the paper says, still leaves DocFS ahead in all four settings, by 1.4× to 5.6×, with the breakdown in the appendix. Hierarchy repair and equation correction call an LLM and a VLM again, so the offline pass is not free on long files.

On DeepSeek, dropping preprocessing falls to 58.7, below the 64.4 from dropping the file system entirely. A bad hierarchy gives the agent a directory that points the wrong way. glob barely fires, so one of the four tools is idle on these benchmarks.

Terms

Source

What people are saying

Related papers

All paper explainers