Step-by-Step Guide to Calculating LSTM by Hand
ProfTomYeh · x · 2026-07-18
Professor Tom Yeh has shared a detailed tutorial on manually calculating LSTM. As recurrent mechanism-based networks like Mamba regain attention, the author guides readers through computing an LSTM cell purely by hand in 15 steps.
Core steps include:
- Initialization: Set the initial hidden state and memory cell.
- Linear Transformation: Stack the current input with the previous hidden state, then multiply by four weight matrices (forget gate, input gate, candidate state, output gate).
- Gating and Memory Update: Apply sigmoid to the first three matrices to get gate values, combine with the tanh-processed candidate state to forget parts of the old memory and write new memory.
- State Output: Multiply the updated memory (passed through tanh) with the output gate to obtain the current hidden state.
- Sequence Progression: Copy the resulting hidden state and memory cell forward, repeating the above steps to process subsequent inputs.
More from Research
- Artificial Analysis chart compares Gemini 3.5 Flash-Lite with 3.6 Flash — Expensive_Syrup_6529 · 2026-07-21
- Gritt raises a new round to automate solar array installation and maintenance — rebeccakaden · 2026-07-21
- NSA's Mike O'Hara: AI Puts Math Research Progress on 'Fruit Fly Years' — AlexKontorovich · 2026-07-21
- Cell study: localized PD-1 engineered T cells reduce brain inflammation in mice — EricTopol · 2026-07-21
- Linear Digressions returns with a new season of audio essays on AI agents — ChrisGPotts · 2026-07-21
- A forecasting lesson on why R-squared alone led to overfitting and worse predictions — mdancho84 · 2026-07-21