ByteDance's FlashMorph2: On-Policy Layer Selection for Hybrid Attention Models

ByteDance-Seed · hf · 2026-11-20

ByteDance Seed introduces FlashMorph2, an on-policy layer-selection method for converting pretrained Transformers into hybrid full/linear attention models.

Improves long-context benchmark scores while selecting layers efficiently across model scales.

Original post →

More from Research

Research channel →