Pope Leo XIV and one of the world’s leading AI companies recently found themselves on opposite sides of a thorny question: Can a chatbot have a conscience?

A New York Times investigation published late last month revealed that Anthropic cofounder Chris Olah proposed withdrawing from a Vatican event to launch Pope Leo XIV’s first encyclical after reading an advance copy, which rejected the possibility of sentient AI. Paragraph 99 of Magnifica Humanitas states bluntly that AI models “do not undergo experiences, do not possess a body, do not feel joy or pain.” Nor do they “have a moral conscience.”

The disagreement reflects a widening debate over whether AI systems could, or should, ever achieve moral personhood. Anthropic spent months consulting religious scholars and ethicists about the possibility of AI consciousness, and about how to instill moral values in its models. But the Vatican—and many secular thinkers—maintain that while AI may imitate human intelligence and moral judgment (a chatbot might generate sound ethical advice without understanding its consequences), genuine morality is unique to the human experience. 

The debate has implications far beyond theology, especially as millions of people already ask chatbots questions that have moral dimensions: They may ask for advice on handling an interpersonal problem at work or caring for an aging parent. Billions of VC dollars are riding on the idea that businesses will increasingly rely on the judgment of AI.

And it’s a fluid situation. AI models continue growing and learning, and demonstrating qualities that were completely unexpected by their creators.

Source link