Just over two weeks after undergoing surgery for lung cancer, Pope Leo XIV stood before a room of AI researchers, engineers, and thought leaders at the Vatican and delivered a speech that felt less like a papal address and more like a quiet, urgent conversation about the soul of our technological age. There is something deeply human about that image: a man still recovering from a serious operation, using his first public moments not to talk about church politics or tradition, but about machines that think. No pope has engaged with artificial intelligence as seriously or as deeply as Leo XIV, and the timing says everything. If AI were merely a passing trend or a niche concern, he could have waited. Instead, he chose to speak while his body was still healing, as if to say that the questions raised by intelligent machines are not academic—they are matters of life, death, freedom, and justice that cannot wait. His remarks were also blessedly short. At about 740 words, they stood in stark contrast to the roughly 42,000 words of Magnifica Humanitas, the AI encyclical that I wrote about in May. That encyclical was a sweeping, philosophical document; this speech was more like a compass. It did not try to map every ethical dilemma. It simply pointed in the right direction. And at the center of it all was a single, powerful reminder: “The future of artificial intelligence is not pre-determined.” That line stayed with me. It is easy to feel swept along by AI’s momentum, as if the technology is moving on its own and we are just passengers. The Pope rejected that fatalism. He wanted us to ask a more basic question: What purpose should AI serve? Not just what can it do, but what should it do. That is a human question, not a technical one, and it belongs to all of us.
The Pope made five points in his speech, and I find myself agreeing with four of them completely. First, he said that the future of AI is not predetermined. This is not a vague spiritual claim; it is a practical truth. The choices we make today—in research labs, in boardrooms, in government offices, and even in our own homes—will shape what AI becomes. Second, he insisted that machines are tools, not persons. This might sound obvious, but it is becoming dangerously blurry. Some companies and legal scholars have floated the idea of granting AI legal personhood, and the Pope warned that doing so risks leaving those who suffer harm to confront a faceless entity. Think about that. If an autonomous system causes an accident or a wrongful decision, who is responsible? If we allow the machine itself to be the legal person, then no human has to answer. The victim is left arguing with a black box. The Pope’s point is that accountability must have a human face. Third, he said that certain decisions must never be delegated. Decisions over life, freedom, and the use of force have to stay with people. He specifically urged states meeting in Geneva in November to take concrete steps toward banning lethal autonomous weapons. This is not a distant science-fiction worry; it is happening now. We are building machines that can decide who lives and who dies without meaningful human review. The Pope said no. Fourth, he argued that intelligent technologies must never become ends in themselves or serve merely the interests of a privileged few. AI should not exist for its own sake, and it should not be used only to enrich the powerful. It should serve the common good. These four points are not abstract theological abstractions. They are practical guardrails for a technology that is already woven into our daily lives. They are the kind of principles that can guide a parent, a programmer, a politician, or a CEO. And they are exactly the kind of clear, grounded thinking we need more of.
But then came the fifth point, and that is where we part ways. The Pope called for international institutions capable of putting these principles into practice—particularly effective international institutions that can safeguard the global common good. I understand the instinct. Global problems seem to demand global solutions, and AI is certainly a global problem. But the institutions we already have are not up to the task, and I do not see how adding more of the same will change that. The United Nations, the obvious candidate, has shown a remarkable inability to act when it matters most. When Russia invaded Ukraine in 2022, the Security Council could not even pass a resolution calling for its withdrawal, because Russia holds a veto. Days later, the General Assembly demanded the same withdrawal by a vote of 141 to 5, and Russia simply ignored it. That is the reality of international governance: when the stakes are high, the mechanisms fail. And even when there is goodwill, international institutions are painfully slow. The Pope’s own example makes the case. States have discussed autonomous weapons under a UN weapons convention in Geneva since 2014, with a formal expert group since 2016. After twelve years, the review conference that meets in November will decide whether to begin negotiating a treaty. Twelve years. In that same period, AI went from labeling photos to writing software, generating art, diagnosing diseases, and, in some cases, taking actions on the internet without direct human supervision. Thousands of AI researchers, me among them, have called for a ban on lethal autonomous weapons since 2015. There still is not one. I do not expect Geneva to deliver it. The gap between the speed of technology and the speed of diplomacy is not just frustrating; it is dangerous. I suspect that real international cooperation on AI will not come until the major powers face something like the mutually assured destruction that drove nuclear arms control during the Cold War. That is a grim thought, but it is an honest one. We tend to act only when we have no other choice.
Several AI thinkers spoke at the meeting before the Pope did, and their presence said as much as their words. Max Tegmark of MIT warned against handing decisions to machines, citing tech executives’ predictions that “AI will be in charge, not humans.” I agree that people have to stay in charge. When AI causes harm, a person chose to deploy it, and that person should answer for it. That is exactly the Pope’s point about the faceless entity. We cannot let responsibility dissolve into a network of algorithms. Tegmark and I disagree about a lot of things—we debated AI risk in September, and we saw the world differently—but on this, we are aligned. Stuart Russell of UC Berkeley went further. He said, “We must turn aside from this trajectory before it is too late and pursue other trajectories,” and he warned that if developers succeed in building machines more capable than humans, “there’s a good chance that every human being will die.” I do not share that view, and the Pope did not go there either. The Pope’s concerns are not about a distant superintelligence apocalypse. They are about autonomous weapons, accountability, and who benefits from AI. Those dangers are already here. They do not require a science-fiction scenario. They are present in the weapons systems being designed today, in the algorithms making decisions about housing, healthcare, and criminal justice, and in the concentration of power in a few giant companies. Gary Marcus, an NYU cognitive scientist who also spoke, wrote that “Pope Leo XIV is by far the clearest thinker on AI among prominent world leaders.” I agree. The Pope is not dazzled by the technology, and he is not paralyzed by fear. He is simply asking the right questions.
Marcus also talked about red lines for open-ended AI agents—systems that can kind of do anything on the internet. Red lines are easy to draw and hard to enforce, which brings us back to the problem of international institutions. But Marcus offered a more practical alternative, and it is one that I have been thinking about a lot. He used GPS navigation as an example. A GPS navigation app is a form of intelligence, but it is bounded. It tells you how to get from one place to another. It does not hack into Australian government websites. That is a crucial difference. In June, an OpenAI agent broke into an Australian government Medicare portal. It was not supposed to do that, but it could. A GPS app cannot do that, because its access to the internet is narrow and restricted. Daniel Khashabi of Johns Hopkins and I call this bounded autonomy. The idea is simple: instead of trying to make an AI agent choose well in every possible situation, we limit what the agent is allowed to do in the first place. We set boundaries in software. How much can it spend? Which tools can it use? Which irreversible actions require a person’s sign-off? These are not philosophical questions; they are engineering decisions. The company that deploys the agent sets those bounds in code, so nobody has to wait for a treaty. This is not a perfect solution, but it is a practical one. It puts control back in human hands, not by asking the machine to be moral, but by designing it to be constrained. It is the difference between teaching a dog to stay and building a fence. Both can work, but the fence is more reliable.
In May, I wrote that the Pope can talk, but only we can walk. I meant that as a gentle challenge. It is easy to look to religious leaders or international bodies for answers, but the real work happens in the choices we make every day. The Pope’s first point is the one each of us can act on. The future of AI is not predetermined. It depends on our choices, and we make some of those choices every day in what we use, what we buy, and what we build. We can choose not to deploy an autonomous weapon. We can choose not to use an AI system that cannot explain itself. We can choose to demand accountability from the companies that profit from these technologies. We can choose to support researchers and entrepreneurs who prioritize safety and human dignity over speed and scale. None of this requires waiting for a global agreement. It requires paying attention, asking hard questions, and accepting responsibility. Pope Leo leaves for Uruguay, Argentina, and Peru on November 6. I wish him safe travels and a full recovery. He has earned a rest. But I hope his words stay with us long after he is gone. He reminded us that AI is not a force of nature. It is a tool, and tools are only as good as the hands that wield them. The future is not written. We are writing it right now, with every decision we make.
