Apple and Duke's LensVLM zooms into compressed pages: 68.9% accuracy at 4.3x effective compression
机器之心 · wechat · 2026-10-14
Apple's foundation models team and Duke University propose LensVLM (NeurIPS 2026), which fixes the one-way-street problem of context compression: the model browses compressed page images covering the full text, then expands only the pages relevant to the question.
Core idea
- Rendering text as images can use fewer vision tokens than the original text tokens, with font size/page height/density as compression knobs — but smaller characters hurt readability: a base model reading 5x-compressed images scores just 31.3% vs. 72.4% for full text.
- A readpage tool returns the original text for text-rendered pages; for PDFs, variants expand to OCR text or high-res pages — on MMLongBench-Doc the high-res variant hits 41.9% vs. 32.1% for OCR, at 10.3x effective compression.
Training
- Built on Qwen3.5-9B-Base: SFT on trajectories generated by Qwen3.5-397B lifts accuracy 31.3%→63.7%; DAPO RL with correctness-based rewards pushes it to 68.9%.
- Across seven benchmarks (NQ, HotpotQA, MuSiQue, HELMET, RULER, Qasper, LongBench): 68.9% at 5x compression, near the 72.4% full-text reference; at 15x compression the base model drops to 18.1% while LensVLM reaches 52.1% (10.1x effective compression).
Efficiency (32K–140K tokens, 15x compression)
- With offline page caching, end-to-end latency falls from 3.62s to 2.52s (30% faster than full-text input); peak KV cache is 1/14 of full text; saturated throughput 3.7x, with 73% lower GPU cost per 1K requests.
- For short contexts, feeding full text remains faster; gains concentrate on long documents and repeated queries reusing cached pages.
Model (apple/LensVLM-9B), code, and a HuggingFace demo are public; Victor Mustar's post drew 280K views.
More from Models
- Opus 5.5 still leads a month later, but open-weight decision models are closing in — airesearch12 · 2026-10-16
- Google restricts Gemini Pro access for Google Plus subscribers, matching free tier — EmergencyLess8861 · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta open-sources Muse Spark: runs on two Mac Studios, license bans passing output off as human-written — mawcopolow · 2026-10-16
- Golden Gate Claude explainer resurfaces: interpretability premise still holds — HarperSCarroll · 2026-10-16
- Hugging Face livestreams release of a new open-weight model — huggingface · 2026-10-16