Model grafting turns Qwen3.5-4B into a causal encoder-decoder, 3.7x faster at 128K prompts
asankhs · reddit · 2026-09-22
A Reddit user demonstrates "model grafting": inspired by DeepSeek-V4.1-Flash's from-scratch causal encoder-decoder architecture, the method retrofits an existing model — cutting at some depth, letting lower layers read the prompt, and injecting upper-layer residual streams as prefix KV via identity-init adapters, then healing with self-distillation from the unmodified parent. The decoder stays unchanged. Details in the author's blog post.
Two variants of Qwen3.5-4B are released on Hugging Face:
- graft8: 3.7x prompt-processing speedup at 128K length, with some accuracy loss.
- graft16: accuracy very close to the parent model, 2.0x speedup.
A reproducible community experiment showing long-context efficiency gains can be added to pretrained models after the fact, with both variants open-sourced.
Related event: Model Grafting Converts Qwen3.5-4B into Causal Encoder-Decoder, 3.7x Faster(2 posts)→
More from Research
- AlphaGenome2 hits SOTA linking distal enhancers to target genes from gradients — anshulkundaje · 2026-11-12
- AlphaGenome2 ablations: early encoder conditioning nearly doubles eQTL recall — anshulkundaje · 2026-11-12
- Author says his CHI2027 ADR paper was better than papers he reviewed — windx0303 · 2026-11-12
- Google launches AlphaGenome 2 to predict how DNA variants affect specific cell types — sundarpichai · 2026-11-12
- Stanford's James Zou joins The Economist podcast on AI agents in science — anshulkundaje · 2026-11-12
- New learning rule boosts Hopfield network memory capacity over 10×, per Nature Communications study — bravo_abad · 2026-11-12