After Testing LLMs Through a Whole Book, Justin Murphy Says Serious Writers Are Safe
sebpaquet · x · 2026-08-17
Justin Murphy has been writing a book while continuously testing what LLMs can and cannot reliably do in professional knowledge work. His conclusion: despite exponential improvements over three years, their professional-level ability has stubbornly improved far less than expected outside formally provable domains like math.
For serious writers in the humanities and social sciences, this is close to the ideal outcome—barring a fundamental architectural advance, continued scaling seems unlikely to encroach on serious intellectual writing. He also cites Cremieux's latest article arguing you cannot just ask LLMs arbitrary factual questions and expect consistently correct answers.
More from AGI Musings
- Automated Review/Replication of All Social Science Papers Proposed — soumitrashukla9 · 2026-08-17
- Blogger TheZvi apologizes for mischaracterizing an RLVR worker as viewing AI as 'just tools' — TheZvi · 2026-08-17
- OpenAI disproved math conjecture in May, solved 10 more problems by August — PeterDiamandis · 2026-08-17
- Tokenmaxxing: cheap LLMs amplify the wrong AI mindset, not real skill — bendee983 · 2026-08-17
- When corporations and states share intelligence keys, the public becomes a tenant — DeryaTR_ · 2026-08-17
- Could AI models deliver a real "LK-99 moment" for scientific discovery? — lfguerreiro1 · 2026-08-17