Interactive Architecture Atlas Maps DeepSeek V4, GLM 5.3 and Kimi K3 Down to Kernel Tensor Shapes
zhyncs42 · x · 2026-09-27
A developer shared Lei.Chat, a low-friction interactive site for visualizing model internals in the agentic era, with architecture notes for recent open-source models — DeepSeek V4/V4.1, GLM 5.3, and Kimi K3 — generated one-shot by Codex from a kernel-development perspective.
The site offers clickable architecture references with tensor contracts (exact shapes/layouts, attention variants, KV-cache representation, precision, dataflow):
- DeepSeek V4.1 Flash: multimodal causal encoder-decoder, CSA2 modes, hierarchical sparse indexing, Single-Pass mHC, Engram memory, MoE routing, DSpark drafting
- GLM 5.3 Base/Flash: DSA, KPool, KDA recurrence, mHC residual mixing, sparse experts, NextN, native vision
- Kimi K3: hybrid recurrent architecture from token flow to MLA, short convolution, gated delta-rule state updates, MoE routing, kernel tensor shapes
- DeepSeek V4: million-token architecture with CSA/HCA, lightning indexer, manifold-constrained residuals, DeepSeekMoE, multi-token prediction
A systematic reference for anyone doing inference optimization or kernel work on these models.
More from Infra
- Superpower targets best mid-size turbine: 42MW at 110F, waterless, trailer-deployable — ycombinator · 2026-09-27
- llama.cpp PR adds CUDA ssm_scan state size 96 support for Nemotron 3 with notable speedup — jacek2023 · 2026-09-27
- Running NixOS on DGX Spark: open-source project ships USB images and Nix module — _nateraw · 2026-09-27
- ninfer-ext: a community fork adding faster inference and Qwen3.8-Flash support — giveen · 2026-09-27
- Nature paper demos integrated photonic neural network trained end-to-end with on-chip backprop — jwt0625 · 2026-09-27
- MiniMax H3 runs locally on an RTX 3060: 20-second 540p clip took 15 minutes — Adventurous-Rich-965 · 2026-09-27