Baidu Open-Sources Unlimited OCR: R-SWA Keeps Memory Constant, Parses 40+ Pages in One Shot
aigclink · x · 2026-10-12
Baidu open-sourced Unlimited OCR (27k GitHub stars), a 3B MoE model (500M active params) fine-tuned from DeepSeek-OCR for end-to-end long-document parsing, deployable locally.
The core trick is R-SWA, mimicking a human scribe's working memory: every generated token always attends to all image visual tokens, but only looks back at the last 128 generated tokens, turning the KV cache into a fixed-capacity queue. Per-token attention cost no longer grows with output length — memory and compute stay constant whether outputting 10k or 100k tokens, and it's 35% faster than DeepSeek-OCR at 6k tokens.
- Each 1024×1024 page compresses to 256 tokens; 32K context parses 40+ pages in one forward pass without missing or repeating lines (Distinct-35 97%)
- Outputs Markdown/JSON directly, handles tables, math formulas and reading order — a good entry point for RAG/knowledge bases
- The mechanism extends to long-sequence generation like ASR and machine translation; vLLM and ms-swift supported, paper on arXiv
Related event: Baidu Open-Sources Unlimited OCR for Long Document Parsing(2 posts)→
More from Models
- Claude Pro poll: 211,770 votes split on keeping 5h limits for 2X weekly usage — koltregaskes · 2026-10-12
- Byteification retrofits token-based LLMs to work at byte level, fixing letter-counting failures — PontiEdoardo · 2026-10-12
- Same Model, Different Harness: OpenCode Refuses Far More Than Codex in Side-by-Side Tests — Current_Balance6692 · 2026-10-12
- User claims GPT's $20 tier may be gone, pricing change suspected — sujingshen · 2026-10-12
- METR's AI time-horizon chart hasn't updated since May, leaving frontier model gains unmeasured — haider1 · 2026-10-12
- AWS open-sources Strands Decider 2B, a laptop-runnable decision model with no text generation — dl_weekly · 2026-10-12