Riemannian–Lorentz Parameter Fusion merges architecturally incompatible vision models without retraining
burkov · x · 2026-10-30
Andriy Burkov covers a model-merging framework addressing a costly limitation: weight-averaging techniques require identical architectures, so global Vision Transformers and linear-scaling state-space models can't be combined directly. Riemannian–Lorentz Parameter Fusion keeps both architectures in a two-branch hybrid system, maps parameters to a common representation, and fuses them geometrically — synthesizing knowledge from pre-trained checkpoints without expensive end-to-end retraining.
More from Research
- Epoch data: leading chips get ~40% more efficient yearly, but AI power draw keeps climbing — teortaxesTex · 2026-10-30
- Soft-MTP: a one-line fix removes noisy gradients from multi-token prediction — pmttyji · 2026-10-30
- Vicki Boykis on ONNX inference for retrieval and ranking: slides from DataPhilly talk — vboykis · 2026-10-30
- Agentic MRI review takes 9th place out of 6299 teams in Kaggle RSNA knee abnormality detection — kagglingdieter · 2026-10-30
- Meta's RoboJEPA: First Scaling Laws for Multi-Embodiment Robotic World Models — apsarathchandar · 2026-10-30
- Agent harness engineering: lessons from 57 papers — same model, 6x performance gap — Marmelab · 2026-10-30