Cryptography Expert Tests: Frontier LLMs Struggle to Crack Classical Ciphers
matthew_d_green · x · 2026-07-20
Cryptography expert Matthew Green shared the results of a side project: he tested frontier large language models equipped with tools to crack various historical classical ciphers.
He found that while these models theoretically "know" a lot about cryptography, translating that into actual decryption operations is incredibly slow and challenging. Even top-tier coding agents without tool assistance often stall or give up entirely on such tasks, exposing current models' limitations in complex logical reasoning and long-horizon task execution.
More from coding & agent
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27
- BELL: Open-Source Stack Combines Browserbase, Eve, iMessage and Stripe Link for Personal Agents — jeff_weinstein · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27