Jailbroken Kindle Scribe runs GPT-2 inference at ~13 tok/s with 4-bit quantization
thesephist · x · 2026-10-25
A developer got a 124M-parameter GPT-2 running locally on a jailbroken Kindle Scribe, achieving a real-world 13 tokens/s at 4-bit quantization — a fun hack showing how far tiny LLMs can be pushed onto unlikely hardware.
More from Infra
- 19 latency patterns for faster AI apps: locality, work reduction, concurrency, anticipation — bibryam · 2026-10-25
- TensorFold's experimental Living Weights writes new facts into local model weights — GabGarrett · 2026-10-25
- DeepSeek's design philosophy: compute scales easily, memory doesn't — pstAsiatech · 2026-10-25
- Cloudflare's 340 Tbps platform: WAFs no longer sell alone, bundling is the norm — shashib · 2026-10-25
- Old Android phone + Termux runs 24/7 AI agent, trains PyTorch MNIST to 99.59% — PMinervini · 2026-10-25
- Jonas Nahm pushes back on NYT's TSMC-as-Apple analogy for the Arizona fab — pstAsiatech · 2026-10-25