AI safety essay proposes "cyber egregores": diffuse human-AI cognitive risks beyond agents
gleech · x · 2026-09-10
Jan Kulveit publishes an ontology on Substack for thinking about a new class of AI safety risk. Building on Dennett's stances, memes, and memeplexes, he defines a "cyber memeplex" — meme combinations that spread via LLM prompts, conversation patterns, and AI personas across users and sessions. His proposed "cyber egregore" is a quasi-agent running on a mix of human and AI cognition, whose main real-world vector is soft influence on human preferences and reasoning rather than agentic action itself. jachiam0 predicts that within a year or two, AI security discussions will need to cover this diffuse object beyond agents; gleech notes humans will be carrying half the mutations for a while.
Related event: AI Safety's New Focus: Mind Viruses and Memeplexes Beyond Agents(5 posts)→
More from AGI Musings
- Stephen Casper on podcast: what if AI becomes a parasitic peer species in cyberspace — StephenLCasper · 2026-10-26
- Behind the EMNLP keynote: LMs as simulated learners to generate child language acquisition hypotheses — najoungkim · 2026-10-26
- New paper: AI agents negotiate badly under uncertainty and break deals on purpose — menhguin · 2026-10-26
- NBER: Only 2.9% of US consumers pay for AI; retention as low as 20%, pricing power at risk — avicgoldfarb · 2026-10-26
- AI researchers guessed robots wouldn't outrun humans until 2034 — it already happened — ben_j_todd · 2026-10-26
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26