Decoder-only OCR Training Revisited with Meta's In-fusion, Better Annotations Planned

wightmanr · x · 2026-11-21

After Meta's in-fusion paper came out, a developer revived his old OCR project: after fixing old bugs, in-fusion proved a natural fit for the model architecture and works. As an alternative to the NaFlexViT encoder + text decoder setup, he trained a decoder-only variant (the original encoder relied on strong pretrained weights, hard to beat with short OCR training).

He trialed with the idl-wds dataset he and @molbap had uploaded to the pixparse project on HF, noting the annotation noise floor is readily apparent — good enough to establish reading capability but better annotations are needed. He's planning agent-driven reprocessing of pdfa-eng, pdfa-multi (never released due to annotation issues), and idl, and has asked Hugging Face for a quota bump as pixparse ran out of Hub space.

Related event: Developer Retrains Small OCR Models Inspired by Meta's In-Fusion Paper(2 posts)→

Original post →

More from Research

Research channel →