"You need a memetic immune system": reasoning alone won't protect you from the smartest wordcel
sebpaquet · x · 2026-09-29
ArtemisConsort argues that a sufficiently smart person can make a compelling verbal argument for any position — she and friends used to do this as a college game. Reasoning alone isn't enough self-defense; you need a "memetic immune system" or you'll be controlled by the most persuasive voice you recently encountered.
The point lands harder in the LLM era, where models' argumentative fluency amplifies this kind of rhetorical manipulation risk.
More from AGI Musings
- Stanford's Code in Place lets students use AI freely — but they must explain their code live each week — chrispiech · 2026-09-29
- Devs debate: product managers may now hold the most valuable skill in the AI coding era — pramodk73 · 2026-09-29
- Gold rush to rebuild everything with AI will collapse for the same reason it exists — nptacek · 2026-09-29
- "If You Told 2020 That AI in 2026 Solved a Millennium Problem, You'd Call It the Singularity" — aran_nayebi · 2026-09-29
- Quintin Pope: cheap finetuning lets AIs defect, making durable AI coordination—and takeover—unlikely — QuintinPope5 · 2026-09-29
- When EA Ambition Means Buying Galaxies and Digital Descendants — abhiadesai · 2026-09-29