Dev tests four tiny OCR architectures inspired by Meta's in-fusion paper

wightmanr · x · 2026-11-21

Developer wightmanr, retraining his old OCR project, found Meta's in-fusion paper a natural fit and ran a series of experiments:

He catalogs four NaFlex-friendly configs, dessert-named: Cruller (cross-attention encoder+decoder, Donut style), Fritter (image tokens as prefix to decoder-only LM), Cronut (single native decoder-only trunk over raw patches + text, optional in-fusion vision weights), and Berliner (Cronut-style with a pretrained LLM trunk like SmolLM/Qwen3 plus in-fusion vision weights).

Related event: Developer Retrains Small OCR Models Inspired by Meta's In-Fusion Paper(2 posts)→

Original post →

More from Multimodal

Multimodal channel →