Geoffrey Irving on the Philosophy of Character Training and Alignment

On August 23, Geoffrey Irving posted a series on the philosophical foundations of character training and AI alignment. His core claims: the circularity problem of character training has philosophical precedent, alignment requires substantial conceptual work in linguistics and moral philosophy, and different AI developers are effectively betting on different moral philosophies.

Confirmed

• Irving noted that early in training, models are weak and poorly aligned; "ethics" is just a number influencing token distributions (e.g. 46318 in new GPTs), raising concerns about how this circular linguistic feedback connects to reality
• He sketched a simplified character-training pipeline: train for a while, tell the model to "be ethical," have it generate its own "ethical" data, keep training, with later steps yet to be determined
• He invoked Quine's critique of logical positivism: human language likewise cannot anchor sentence-by-sentence to experiments as a DAG, but the whole language network can coherently anchor to reality as a whole — offering a lens for AI models' circular language
• He argued that even if theoretical models of alignment exist, humans need substantial conceptual work to discover them; character training connects deeply to philosophy of language and moral philosophy
• He observed that different developers bet on different moral philosophies: Anthropic leans toward virtue ethics, OpenAI closer to deontology (both mixed with other elements)

Why it matters

• Character training is a mainstream alignment practice; its philosophical legitimacy directly affects the credibility of self-generated-data reinforcement strategies
• Different moral theories may extrapolate to superintelligence in radically different ways, so which theory is better shapes the values of future advanced AI

2026-08-23 ~ 2026-08-23 · 7 related posts

• [source] Conceptual Work in AI Alignment: The Link Between Character Training and Philosophy — geoffreyirving · 2026-08-23
• A Rough Character Training Story: Self-Generating Ethical Data — geoffreyirving · 2026-08-23
• [source] Geoffrey Irving on the Grounding Problem in Character Training and Alignment — geoffreyirving · 2026-08-23
• Drawing Parallels to Quine: The Loopy Nature of Human and AI Language — geoffreyirving · 2026-08-23
• Quine's Argument: The Entire Web of Language Grounds into Experiment Coherently — geoffreyirving · 2026-08-23
• [source] AI Developers Bet on Different Moral Philosophies: Anthropic on Virtue Ethics, OpenAI on Deontology — geoffreyirving · 2026-08-23
• Different moral theories shape superintelligence differently; Anthropic leans virtue ethics while OpenAI favors deontology — geoffreyirving · 2026-08-23