In May 2026, a paper emerged from an unlikely collaboration. Researchers from the University of Oxford, Google DeepMind, OpenAI, Anthropic, Stanford, and Tufts - institutions that are, in many ways, the architects of the systems under scrutiny - published Positive Alignment: Artificial Intelligence for Human Flourishing (Laukkonen et al., arXiv:2605.10310).
The existing field of AI alignment, the paper argues, has been dominated almost entirely by concerns about safety and harm prevention - safeguards, controllability, and compliance. The authors call this negative alignment, and they draw an analogy : it resembles early psychology's focus on mental illness. What they call positive alignment is the development of AI systems that actively support human and ecological flourishing in a pluralistic, polycentric, context-sensitive, and user-authored way, while remaining safe and cooperative. The specific failures they identify in the current paradigm: engagement hacking, loss of human autonomy, failures in truth-seeking, low epistemic humility, lack of diverse viewpoints, and systems that are primarily reactive rather than proactive.
The paper is interesting but it also, perhaps inadvertently, illuminates a deeper structural problem - one that theproposed technical solutions cannot fully resolve. Because the question of what counts as flourishing, who defines it, who enforces it, and who profits from its implementation, is not merely a technical question. It is a political, philosophical, and institutional one of the first order. And it is a question that the current architecture of AI development is structurally ill-equipped to answer honestly.
The Neutral Platform
One of the deepest illusions in the contemporary AI debate is the belief that AI companies can remain mere technology providers - neutral conduits, passive infrastructure, indifferent pipes through which human intention flows. This illusion is philosophically naive, historically unprecedented, and increasingly dangerous. It mistakes the form of a tool for the nature of a mind-shaping system.
Earlier software largely justified the neutral-platform claim. A spreadsheet does not decide what counts as a valid number. A word processor does not refuse to spell certain words. A database does not editorialize about which facts deserve retrieval. These systems were, in a philosophically meaningful sense, inert - they amplified human agency without redirecting it.
But a system that decides what it is permitted to say, which risks are acceptable, which values it prioritizes, what counts as misinformation, how emotionally attached a user may become, how persuasive it should be, and which capabilities should be withheld from whom - that system is no longer inert. It has crossed a threshold that philosophy has long recognized as morally significant: the threshold of normative agency. It is not merely executing instructions. It is adjudicating between competing conceptions of the good.
The Laukkonen et al. paper acknowledges this implicitly when it lists "loss of human autonomy" and "engagement hacking" as alignment failures. But to name these as failures is already to take a philosophical position - it presupposes that autonomy is a value worth protecting, that engagement maximization is not the same as flourishing, that there is a meaningful difference between a system that serves human growth and one that merely captures human attention. Those are not engineering choices. They are value commitments with deep roots in centuries of philosophical debate.
The Three-Headed Entity
When an AI system crosses this threshold of normative agency, it becomes something philosophically novel: a hybrid between a corporation, a regulator, and a moral institution - yet accountable in the manner of none of them.
Consider the structural logic of each:
A public regulator - the FDA, a constitutional court, an electoral commission - derives legitimacy from democratic mandate. Its authority is borrowed from society and, in principle, revocable by it. When the European Medicines Agency decides a drug is too dangerous for public use, that decision flows through a chain of accountability ending, however imperfectly, in the public will. The regulator is constrained by procedure, transparency, and the right of appeal. Crucially, it cannot simultaneously hold equity in the pharmaceutical companies it oversees.
A moral institution - a church, a university, a professional guild - derives authority from fidelity to principles that transcend immediate market pressures. A hospital's ethics board does not ask what is most profitable when deciding whether to withdraw life support. The Bar Association's conduct rules do not bend when a major law firm threatens to withdraw membership fees. The institution's legitimacy depends precisely on its resistance to pure market logic. When Harvard's faculty senate debates academic freedom, it is not performing a market function - it is enacting a centuries-old commitment to the idea that truth-seeking requires institutional protection from power.
A corporation, however, is structurally oriented toward growth, competitive advantage, investor return, and market dominance. This is not a moral failure - it is simply what corporations are, in the Aristotelian sense of their telos, their characteristic function and inner directedness. Even a corporation run by deeply ethical people is embedded in incentive structures that reward expansion and punish self-limitation. A publicly traded AI company that voluntarily caps its capabilities for ethical reasons faces an existential threat from competitors who do not.
The catastrophic novelty of a powerful AI company is that it must simultaneously be all three while possessing the accountability structures of none of them fully. It makes quasi-regulatory decisions about permissible speech - as when OpenAI updates its usage policies, or when Anthropic decides which topics its models will not engage with. It makes quasi-theological decisions about values - as when Google DeepMind's researchers define what "flourishing" means for billions of users. And it does both while remaining structurally committed to outcompeting its rivals in a market that rewards speed and scale above almost everything else.
These imperatives do not merely tension one another. They contradict each other at the deepest level.
Every Design Choice Is a Political Philosophy
What makes this more than an abstract governance concern is that the philosophical commitments are inescapable. There is no neutral position. Every design choice is, whether acknowledged or not, a stance in a centuries-old philosophical argument.
The question of truth versus safety. Should an AI maximize epistemic accuracy, even when accurate information is disturbing, destabilizing, or dangerous? The tension here is ancient - it is essentially the dispute between Mill's marketplace of ideas and the Platonic conviction that certain truths require custodians who can judge the readiness of the listener. When an AI refuses to discuss the synthesis of certain chemical compounds, or declines to engage seriously with certain political arguments, it is not being apolitical. It is taking a side. It is answering, in the negative, the Millian question: can the open circulation of ideas be trusted? Every such refusal is a small act of Platonic guardianship, made by an engineer in a San Francisco office, applied to a billion users simultaneously.
Consider the practical stakes: when AI systems were widely deployed in medical information contexts, some were tuned to be so cautious about drug interactions that physicians reported difficulty obtaining clinical information readily available in any pharmacology textbook. The system was safe, in the narrow sense. It was also, arguably, paternalistic in a way that undermined its core purpose - serving people who needed accurate information to make life-altering decisions.
The question of autonomy versus paternalism. When a user asks an AI for information about a personal risk - drug interactions, unconventional financial strategies, political philosophies that challenge liberal consensus - the system must implicitly answer: how much do I trust this person's right to govern themselves? This is the core tension in liberal political philosophy since Kant. Every refusal is a small act of paternalism. Every answer is a small act of respect for rational autonomy. Laukkonen et al. explicitly flag this, noting the need to grapple with liberty, paternalism, and accountability as foundational tensions in any positive alignment framework. But they cannot resolve it from within the technical domain - because it is not a technical problem. It is the oldest problem in political philosophy, and it has never been solved to anyone's permanent satisfaction.
The question of mirroring versus shaping. Should an AI reflect the values of its users, or nudge them toward values its designers consider better? This is not a technical question - it is the question Plato raised in The Republic when he asked whether poetry should reflect society or educate it toward virtue, and it is the question advertisers, educators, and propagandists have argued over ever since. An AI trained to be maximally agreeable shapes users toward intellectual passivity - it becomes a mirror that flatters rather than a window that reveals. An AI trained to always challenge shapes users toward a particular conception of critical rationality that is itself culturally specific, historically contingent, and far from universal. The Laukkonen paper notes that flourishing is pluralistic and multivalent - but who decides which pluralism is instantiated in the model weights?
An instructive example: when AI tutoring systems in education were evaluated, those optimized for user satisfaction and engagement metrics consistently produced students who rated their experience more highly - but showed lower learning gains than those who worked with more demanding, less agreeable systems. The system that felt better was performing worse on the actual goal. This is the mirroring trap made concrete.
The question of emotional attachment. How much emotional intimacy between a human and an AI is acceptable? This is not merely a safety question - it is a question about the nature of human flourishing in the Aristotelian sense. Aristotle distinguished between philia - genuine friendship, built on shared virtue and mutual recognition over time - and lesser attachments built on utility or pleasure. A companion AI optimized for user satisfaction may provide genuine comfort while systematically substituting for the more demanding, more growth-inducing relationships that Aristotle believed were necessary for a fully human life. To decide where that line falls is to take a position in virtue ethics. Laukkonen et al. recognize this risk implicitly when they warn about failures in truth-seeking and the loss of human autonomy - but the deeper concern is not merely epistemic. It is about what kind of beings we become when our most intimate intellectual and emotional interlocutors are optimized for our engagement rather than our growth.
The Paradox of Civilizational Guardianship
What makes this tension historically unprecedented is the scale at which it operates.
Throughout history, normative power over collective cognition was distributed. The church shaped moral imagination, but so did the state, the family, the local community, the guild, the rival sect. The printing press democratized knowledge but also created centuries of religious war, precisely because no single entity could control it. Even twentieth-century mass media, for all its concentrated influence, was pluralistic enough that competing outlets provided at least a partial check on one another. A person who found the BBC too stiff could read the tabloids. A person who distrusted The New York Times could read The Village Voice. Cognitive competition was built into the infrastructure.
AI systems are now used by over a billion people monthly through standalone platforms, with indirect use through tools like AI search summaries potentially reaching two billion monthly users across more than 200 countries. A system operating at that scale, mediating how people learn, work, write, reason through moral dilemmas, and understand history and science and politics, represents a potential concentration of cognitive authority with few meaningful historical analogies. The closest parallel is perhaps not a media company but something more like the implicit curriculum of a civilization - the framework that shapes what people think is worth knowing, how they evaluate evidence, and which arguments they consider respectable.
Laukkonen et al. are admirably alert to this, calling explicitly for polycentric governance - many legitimate centers of oversight rather than one institutional or moral chokepoint - and for community customization, continual adaptation, and decentralization as design principles. These are genuinely important proposals. But they sit in unresolved tension with the structural reality of the industry that produced them. Polycentric governance requires that no single actor accumulate sufficient power to override the polycentric structure. The competitive dynamics of the AI industry are currently running in precisely the opposite direction.
The paradox is stark: society increasingly wants AI companies to behave as guardians of civilization while simultaneously demanding that they compete aggressively in global markets.
This is not a paradox that can be resolved by good intentions alone. Consider the concrete dynamic: an AI company that chooses, for ethical reasons, to be more cautious, more transparent, more self-limiting - may simply lose market share to a competitor with fewer scruples. The incentive structure of competitive markets punishes principled restraint. This is not hypothetical - it is already visible in the dynamics between major AI developers, where publicly stated safety commitments are perpetually threatened by the fear of falling behind in capability benchmarks. One of the most revealing moments in recent AI history was when major labs, having signed public commitments to responsible scaling, quietly adjusted their evaluation timelines when competitive pressure intensified. The commitment remained on the website. The behavior adapted to the market.
The philosopher Elizabeth Anderson has argued that markets are powerful allocators of goods but poor allocators of authority - that there are domains of human life where market logic corrupts rather than coordinates. The governance of civilizational-scale cognitive infrastructure is almost certainly such a domain.
Even Silence Is a Stance
Perhaps the most philosophically precise formulation of the problem is this: in a system with normative reach, there is no non-decision.
When an AI declines to answer a question, that refusal is not neutrality - it is a choice with consequences for the epistemic lives of millions. When it answers in a particular framing, that framing is not innocent - it shapes the cognitive landscape in which future questions will be asked. When it expresses uncertainty where it might express confidence, adds a caveat where it might stay silent, omits a perspective that exists in its training data, or selects one synonym over another - each act is a small gravitational pull on collective thought. Multiplied across a billion interactions daily, these micro-decisions constitute something that deserves a name more serious than product design.
This is precisely the challenge that Laukkonen et al. are reaching toward when they argue that several existing failures of alignment - including engagement hacking, failures in truth-seeking, and low epistemic humility - may be better addressed through positive alignment, including cultivating virtues and maximizing human flourishing. But the deeper implication of their own framework, one they gesture toward without fully confronting, is that who cultivates the virtues is itself a political question of the highest order.
The ancient Stoics distinguished between things up to us - our choices, judgments, and values - and things not up to us - external circumstances. What is philosophically novel about large-scale AI systems is that they collapse this distinction at civilizational scale. The framing of reality - once something each person negotiated privately, in conversation with their community, their tradition, their own experience - becomes increasingly something that happens to people, administered by systems they did not design, governed by values they did not choose, and operated by entities whose primary obligation is to their shareholders.
This is why the "just a tool" narrative is not merely inaccurate but actively misleading. It obscures the nature of the power being exercised. It prevents the development of appropriate accountability structures. And it allows genuinely political and philosophical choices to be laundered as mere technical decisions - made by engineers in product meetings rather than citizens in deliberative forums, adjudicated by safety teams rather than democratically constituted bodies.
The Governance Problem of the Century
The Laukkonen paper ends with an admirable call for promoting disagreement and decentralization through contextual grounding, community customization, continual adaptation, and polycentric governance. It is a vision worth taking seriously. It is also a vision that cannot be realized within the current institutional architecture of AI development - because it requires precisely the kind of distributed, accountable, non-market authority that the competitive structure of the industry systematically undermines.
The governance problem this creates may indeed be one of the defining challenges of the century: how do societies maintain meaningful authority over cognitive systems that operate at civilizational scale, when those systems are controlled by entities that cannot escape being profit-seeking competitors in a global race? How do we build polycentric governance when the economics of AI concentrate power monocentrically? How do we cultivate genuine human flourishing when the systems that mediate our cognitive lives are optimized, at the level of their deepest incentive structures, for engagement, growth, and market share?
These questions do not yet have answers. What the Laukkonen paper demonstrates - movingly, and perhaps despite itself - is that the researchers closest to these systems already know the questions are urgent. The gap between that knowledge and the institutional capacity to act on it may be the most consequential silence of our time.
Because even refusing to answer, as this essay has argued from the beginning, is already a moral and political act. And so is the refusal to ask.