Redditor Compiles Mega List of Open-Source LLM Inference Optimization Projects and Papers
Dramatic-Chard-5105 · reddit · 2026-09-04
A Reddit user compiled a comprehensive list of recent open-source projects and papers improving open LLM efficiency and accessibility. Hardware/inference projects include colibri (unified VRAM+RAM+storage memory hierarchy for MoE), exo (Mac clustering), MLX distributed-inference experiments, Houmo's DRAM-PIM targeting >1TB/s and 3× efficiency, and d-Matrix 3DIMC. Papers cover edge-distributed MoE inference (WDMoE, MDI-LLM), on-demand expert loading (OD-MoE: 99.94% expert-prediction accuracy at 1/3 GPU memory), and speculative decoding with expert prefetching (MoE-SpeQ up to 2.34×, SP-MoE, MoE-Spec).
More from Infra
- Micron's G11 NAND doubles density vs G9 with 400+ layers, volume production in 2H 2027 — rohanpaul_ai · 2026-10-24
- PSA: Streaming n-grams from an SSD causes real wear via read disturbance and reclaim — Chromix_ · 2026-10-24
- ISTA quantizes DeepSeek-V4.1-Flash to 2 bits with zero benchmark loss, runs on one 12GB GPU — Loginhe · 2026-10-24
- Nebius Token Factory launches cached tokens with automatic prefix reuse and discounted billing — HowDevelop · 2026-10-24
- Pedro Domingos: Sam Altman's AI race strategy is cornering the compute market — pmddomingos · 2026-10-24
- Ex-Fidelity manager: CoreWeave is a zero short of a bailout, AI is Shale 2.0 — SumitGup · 2026-10-24