The first LLM? Markov hand-computed letter probabilities in 1913
jxmnop · x · 2026-09-13
A fun AI-history thread: the poster argues the first "LLM" was built by Andrey Markov in 1913 — he tallied 20,000 letters from a famous novel and manually computed conditional probabilities p(vowel|vowel), p(consonant|vowel), etc., essentially hand-training a bigram model. The quoted reply adds context: Jeff Dean trained an n-gram model on the entire internet in 2007, Jelinek coined "language model" in the 1970s, and Claude Shannon was estimating English entropy back in 1951 — hence Anthropic naming its model Claude.
More from Fun
- 'I'm Not Decelerating!': AI Scaling Meme Resurfaces — McDonaghMatthew · 2026-09-13
- Dev jokes about draining his 401(k) to buy GPUs after reading Dario's essay — generativist · 2026-09-13
- Revealing I spent a couple grand on LLM subscriptions made everyone stop talking to me — henloitsjoyce · 2026-09-13
- PufferLib's satirical announcement: puffers to form a supermassive black hole by 2048 — jsuarez · 2026-09-13
- AI image claims the decayed throne of art in the name of Dadaism — Independent_Fan_3915 · 2026-09-13
- ezyang: Kolmogorov complexity, but for LLM prompts — ezyang · 2026-09-13