Tsinghua's Meta Expert: Cross-Layer Shared Experts Match MoE With 70-80% of Experts
TsinghuaC3I · hf · 2026-10-23
Researchers from Tsinghua University and Shanghai AI Lab propose Meta Expert, replacing layer-local experts in MoE models with cross-layer shared meta-experts. Empirical comparisons show lower parameter redundancy, lower cross-entropy loss, and higher downstream accuracy; with activation count and granularity fixed, models match Vanilla MoE performance using only 70-80% of the experts, substantially improving storage and retrieval efficiency.
Related event: Meta Expert: cross-layer expert sharing matches MoE with 70-80% experts(3 posts)→
More from Infra
- llama.cpp team launches Llama: one-click local AI on your Mac menu bar — Hugging Face · 2026-10-23
- China's daily token use hits 500T, up 4x as average price doubles to ¥13.46 — teortaxesTex · 2026-10-23
- New all-reduce kernels deliver 12% decode speedup for DeepSeek-V4.1 on PCIe-connected GPUs — AlpinDale · 2026-10-23
- Offshore oil rigs could be converted into AI data centers powered by stranded gas — clemnt · 2026-10-23
- Same native UI, two stacks: Swift build is ~80MB Mac-only, Zig build is ~10MB cross-platform — jedisct1 · 2026-10-23
- SurfSense: an open-source local NotebookLM with MCP agents and a prompt-caching lesson — FurtiveMirth · 2026-10-23