llm-blocks: Swap MLA, Mamba-2, MoE blocks in your LLM with one line of code
cneuralnetwork · x · 2026-11-17
- Developer cneuralnetwork released llm-blocks, an Apache-2.0 open-source library offering LEGO-style composable, swappable building blocks for LLM architecture research in PyTorch.
- Swap attention for MLA, Mamba-2 or Gated DeltaNet, a dense FFN for MoE, or plain residuals for hyper-connections by changing one line, while the rest of the model, training loop and generation stay unchanged.
- Example code combines GQA+RoPE, SwiGLU and RMSNorm via nn.Block, and summary() prints per-layer parameter breakdowns, active params and FLOPs/token.
More from coding & agent
- Five field-tested tips to get Claude to design without AI slop — nateparrott · 2026-11-17
- A Pharma Agent That Flags Developability Risks on Every Sequence Registration — nlarusstone · 2026-11-17
- 55lb drone built with Claude Opus in 6 days — and then it crashed — IgorCarron · 2026-11-17
- Google ships Antigravity slash commands cheat sheet for modes, planning and automations — rseroter · 2026-11-17
- YC F26 Startup Maritime Launches a Managed Cloud for Running AI Agents — ycombinator · 2026-11-17
- Premiere AI Editing Test: Same 12-Min Interview Took 48 Min With AI vs 33 by Hand — BlackGull · 2026-11-17