Crypto Expert Tests LLMs on Classical Ciphers, Revealing AI Toolchain and Dev Paradigm Shifts

Cryptographer Matthew Green recently launched a side project to test the practical abilities of frontier large language models (LLMs) and coding agents in cracking historical classical ciphers. The tests not only revealed the current potential and limitations of AI in cryptanalysis but also unexpectedly exposed deep shortcomings in existing AI toolchains, sparking a profound discussion about a shift in future software development paradigms.

Practical Details and Capability Boundaries

Green found that although LLMs possess vast cryptographic knowledge, they struggle to translate it into actual decryption operations. Without external tool assistance, even top-tier coding agents get bogged down in slow attempts and eventually give up. The current viable workaround is to brute-force the creation of dedicated tools for specific ciphers, strictly constraining the agent to multi-hypothesis testing and self-scoring. After extensive development using Claude and Codex, he concluded that AI programming tools excel at cracking substitution, homophonic, and some polyalphabetic ciphers, making them some of the best tools available today, though this approach remains difficult to generalize.

Toolchain Shortcomings and Agent Preferences

In practice, Green observed that current AI coding agents act blindly when handling specific tasks, often randomly scraping poor-quality open-source projects from GitHub and messing around until they succeed or fail. He criticized the local MCP server architecture favored by current AI designers, arguing it lacks internal testing (dogfooding) and is too clunky. Due to supply chain risks and usability hurdles, AI agents are wary of running local servers with broad capabilities and instead prefer directly downloading and running command-line interface (CLI) tools.

Software Development Paradigm Shift

Based on these pain points, Green highlighted a notable new trend: the primary consideration for software developers is shifting from "human comfort" to "AI agent comfort." For instance, adding instructions for agents in a README actually triggers security warnings that deter them. He believes the future goal is to build a high-quality tool ecosystem that agents can autonomously discover and invoke. This view was echoed by other practitioners: @TheZachMueller shared his work at Lambda, exploring how to make systems more agent-friendly when needed; @FinancialAd7297 noted that as AI becomes ubiquitous, software interfaces will increasingly cater to agents, emphasizing APIs, structured data, and permission controls, with humans shifting more into supervisory roles.

2026-07-19 ~ 2026-07-20 · 13 related posts

• Lambda is Designing for Agent Friendliness — TheZachMueller · 2026-07-19
• Software Increasingly Designed for Agents — Financial_Ad_7297 · 2026-07-20
• Testing Frontier LLMs on Classical Cipher Breaking — matthew_d_green · 2026-07-20
• Cryptography Expert Tests: Frontier LLMs Struggle to Crack Classical Ciphers — matthew_d_green · 2026-07-20
• [source] Limitations of LLMs and Coding Agents in Cryptanalysis — matthew_d_green · 2026-07-20
• AI Agent Cryptanalysis Exposes Flaws in MCP Toolchains — matthew_d_green · 2026-07-20
• Cryptographer Highlights Flaws in AI Coding Agent Toolchains — matthew_d_green · 2026-07-20
• Flawed Local MCP Server Design: AI Agents Need Better Tool Discovery — matthew_d_green · 2026-07-20
• [source] Cryptographer Criticizes Local MCP Server Design — matthew_d_green · 2026-07-20
• AI Agents Prefer Local CLI Over MCP Servers — matthew_d_green · 2026-07-20
• [source] Building Software for AI Agents: A New Paradigm for the Future — matthew_d_green · 2026-07-20
• Programming for AI Agents: Software Pivots to Machines — matthew_d_green · 2026-07-20
• AI Models Excel at Breaking Classical Ciphers — matthew_d_green · 2026-07-20