How to Count AIs: Individuation and Liability for AI Agents
Yonathan Arbel, Peter Salib, Simon Goldstein
cs.CY, cs.AI
2026-02-25
When AI agents cause harm, the first question is which AI did it. The 'Algorithmic Corporation', a human-owned AI-run entity, solves tracing actions and distinguishing agents.
The authors start from a premise: before long, vast numbers of AI agents will act autonomously in the economy, and things will go wrong. But to assign liability, the first hurdle is not whom to penalize, it is "which AI did it." AIs have no bodies; a single conversation may span many chips. A single "agent" is often a temporary assembly of multiple models and instances, and agents can copy, split, merge, swarm, and vanish at will. Before any enforceable liability scheme can attach to AI actions, agents must first be individuated and traceable.
The authors split identity into two layers. Thin identity ties each AI action to a human principal, akin to KYC in finance. Thick identity sorts millions of AI entities into stable, coherent, persistent units, because agents make their own decisions and pursue their own goals (alignment is unsolved), and the law sometimes has to incentivize or constrain the AI itself, which requires telling it apart from others. Most existing legal scholarship touches only thin identity.
They propose the "Algorithmic Corporation" (A-corp): a legal-fictional entity that, like a corporation or LLC, can hold property, make contracts, and be sued, is owned by humans, but is designed to be run by AIs. Two core components are legal-fictional personhood and secure governance infrastructure. A-corp owners must be publicly registered, and ownership changes take effect only when recorded. Every A-corp action carries a cryptographic certificate, analogous to the TLS/SSL public-key system, verified in milliseconds against a public registry that stores public keys. A senior AI manager holds the master certificate and can grant fine-grained, limited-scope permission tokens to subordinate AIs.
Thin identity is solved by tracing: any certified action leads back to a registered human owner. Thick identity rests on a resource-constraint thesis: to pursue a goal, an AI needs resources, especially compute, and the A-corp is the vessel through which it holds and uses them. AI managers therefore have a strong incentive to share control only with goal-aligned AIs, since a misaligned manager could expropriate assets; A-corps with poor internal governance are outcompeted, run out of resources, and "die." In equilibrium, A-corps self-organize into coherent entities that respond rationally to legal incentives.
This is a law and governance paper with no experiments; the argument runs on analogy and mechanistic reasoning. Its key claims are that thin and thick identity are distinct problems and existing work mostly addresses only the former; that AI individuation has six characteristic difficulties (swarming, cross-model coordination, Ship-of-Theseus drift in weights and prompts, rapid creation and destruction, copying and branching, and observability limits); and that A-corps sidestep the philosophical puzzles via resource constraints and selection, letting stable, goal-coherent entities emerge without first resolving whether an AI is a person. Implementation can proceed sub-legally, through new legislation, or under existing corporate law. The authors argue voluntary adoption will be insufficient and mandates at choke points such as compute and market access are needed. The idea also bears on alignment: reinforcement learning and constitutional AI could train A-corps rather than model instances, aligning training objectives with legal liability.
For those building agents, the paper sharpens an underappreciated premise: for agents to be deployed at scale, trusted, and held accountable, there must first be a way to count, separate, and trace them. The A-corp offers a concrete, engineerable governance shell (cryptographic certificates plus a registry). It also points at product design: giving an agent a clear asset boundary and a verifiable identity is not only compliance, it is an economic lever for constraining its behavior.
This is a normative proposal, not an empirical study, and many of its mechanisms (especially the claim that goal-coherent A-corps emerge in equilibrium) rest on strong assumptions, such as AI managers reliably evaluating subordinates' goals and resource competition sufficing to weed out poorly structured entities. The paper discusses collusion, deceptive alignment, and one-shot catastrophic harms but cannot be said to cover them fully. It is framed in US corporate law, and transfer across legal systems needs separate argument. Privacy, security, and abuse risks of the registry are not developed. Overall it reads as a thought experiment, still far from deployment.