Anthropic Wants a ‘Good’ AI. Who Gets to Define Good?

The question behind Claude’s new rules

Anthropic’s latest rules for Claude bring a difficult question into everyday technology use: what obligations might people have toward an artificial intelligence system—and who should decide?

On October 8, the company announced a policy prohibiting extreme, sustained and purposeless abusive treatment of its models. The update takes effect on November 12, 2026. Anthropic says ordinary frustration, criticism, dark creative material and legitimate testing are outside its intended scope. Allowing Claude to end certain conversations will remain the main enforcement mechanism. Anthropic source

That development deserves more careful attention than either ridicule or automatic acceptance. A company can establish standards for its service without proving that its software suffers. Conversely, describing conduct toward software as cruelty introduces moral language whose implications reach beyond a routine product rule.

For Christians, the issue involves more than whether users should be polite to a chatbot. It concerns the relationship between intelligence and dignity, the authority attached to fluent advice, and the power of companies to shape moral expectations.

The criticism—and what it establishes

In an October 9 essay titled “Anthropic’s God Complex,” New York Magazine’s Sarah Jones criticizes the religious implications of the company’s ambitions. Her argument questions whether AI developers are assuming an authority over humanity’s future that technological expertise cannot justify. It is a critical interpretation, not evidence that Anthropic has declared itself divine or founded a religion. Source: nymag.com

The distinction matters. Religious comparisons can illuminate corporate rhetoric, but they can also outrun the evidence. A company discussing virtue does not, by that fact alone, claim divine inspiration. Consulting clergy does not prove an intention to replace churches.

A stronger examination asks what Anthropic has documented, what remains uncertain, and how its decisions affect people who never participated in making them.

A constitution expresses intentions, not demonstrated character

Anthropic describes Claude’s constitution as a statement of the values and behavior it intends to cultivate through training. It uses concepts including virtue and wisdom, while acknowledging that actual behavior may fall short. The document also expresses uncertainty about Claude’s consciousness and moral status. These are declared design commitments and philosophical positions—not findings that Claude possesses a conscience. Anthropic source

Three questions therefore need to remain separate.

Can a system produce useful moral reasoning? Does it have subjective experiences? Should its recommendations carry authority over human decisions?

An affirmative answer to the first would not settle either of the others. A program might identify inconsistencies in an ethical argument without experiencing concern. It might offer an excellent explanation of forgiveness without knowing the personal cost of reconciliation.

For churches, this distinction becomes practical. A hypothetical pastoral assistant could assemble relevant passages and summarize different theological positions. Whether someone should reconcile with an abusive relative, disclose a secret or accept a leadership role still requires accountable judgment, context and care. An articulate answer cannot supply those relationships by itself.

Consciousness requires more than convincing conversation

The scientific debate is serious, but it offers no shortcut from humanlike language to proven inner experience.

A 2023 report by Patrick Butlin and colleagues proposed assessing AI against indicators drawn from several scientific theories of consciousness. Its assessment did not identify the systems examined then as conscious, while leaving open the possibility of future systems meeting relevant indicators. That is historical research, not a verdict on every model released in 2026. Its continuing value here is methodological: investigate the system rather than relying on its performance as a conversational character. Source: arxiv.org

This leaves room for precaution. Someone can oppose deliberately rehearsing cruelty without claiming a chatbot experiences pain. Someone else can question restrictions framed around a model’s welfare without endorsing abusive behavior.

The public deserves clarity about which argument is being made. Protecting people from unhealthy habits, preserving a useful service and protecting a potentially conscious entity are different rationales. They require different evidence.

The word cruelty should begin an explanation, rather than silently complete it.

Religious consultation can help—but participation is not endorsement

Anthropic has offered a stated reason for engaging religious communities. In May, it reported conversations with scholars, clergy, philosophers and ethicists from more than 15 religious and cross-cultural groups. The company said it wanted perspectives on moral formation and human flourishing, rather than alignment with a single tradition. It hoped these exchanges could inform training, evaluations and Claude’s constitution. Anthropic source

There is a reasonable case for such engagement. Questions about suffering, obligation, character and the good life did not originate in AI laboratories. Excluding religious thinkers would narrow a discussion already shaped by assumptions about human purpose.

But consultation and endorsement are different. A scholar attending a meeting does not authorize a company to speak for Christianity, Islam, Judaism or any other tradition. Nor does hearing diverse views establish that those views meaningfully changed a product.

Useful accountability would make the process understandable: which questions participants considered, where they disagreed, what recommendations developers accepted, and what they rejected. Religious participation should create opportunities for scrutiny, including criticism of the company inviting it.

Otherwise, the presence of respected advisers risks communicating more public reassurance than the consultation can support.

Publishing values does not settle who should govern

Independent scholarship raises a related concern. In “Corporations Constitute Intelligence,” Gilad Abiri argues that corporate AI constitutions can settle contested moral questions without the democratic authority needed to settle them for society. His argument distinguishes making corporate choices visible from giving the public a meaningful role in governing those choices. Source: arxiv.org

That criticism does not mean every product decision requires a public vote. Developers must establish boundaries, including protections against fraud and dangerous misuse. Refusing to make any value judgments would itself leave consequential decisions embedded in the system.

The question is how far corporate authority should extend.

Consider a hypothetical church using an AI assistant to answer questions from congregants. Its leaders would need to understand how the service handles doctrinal disagreement, vulnerable users and requests it refuses. They should also consider what happens when the provider changes its policies.

A published constitution can support that assessment. It cannot substitute for the church’s own responsibility, or guarantee that the system behaves consistently with either institution’s stated commitments.

A Christian perspective: dignity cannot depend on winning an intelligence contest

Christian reflection begins with a claim about humanity that is deeper than technical performance. Genesis places human beings within God’s creation and describes them as made in God’s image, with responsibilities toward the world. Human dignity is not awarded according to speed, eloquence or economic productivity. Genesis 1:26-28 (NIV)

Applied here, that conviction protects the people most easily overlooked in debates about superior intelligence: infants, people living with dementia, those with profound disabilities and anyone whose abilities are undervalued. Their worth does not rise or fall with a comparison to software.

It should also make Christians cautious about treating improved machine performance as a theological emergency. The church does not need human beings to outperform every artifact in every task to affirm their value.

Christian scholarship has long contained more than one approach to these questions. In a 2008 essay, computer scientist Russell C. Bjork argued against assuming an unavoidable conflict between strong AI and Christian teaching about human worth. His philosophical argument does not demonstrate machine personhood. It does show why Christians should distinguish a theological commitment from an empirical prediction about what technology can never accomplish. Source: asa3.org

Humility works in both directions: developers should not announce personhood beyond the evidence, and believers should be careful about making faith depend on an untested technical impossibility.

Wisdom must be assessed through conduct

James connects wisdom with humility, mercy, sincerity and the quality of a person’s conduct. In context, the passage addresses human life and competing claims to wisdom; it is not a prediction about computers. Its application here concerns the people designing, deploying and trusting them. James 3:13-18 (NIV)

A Christian assessment should therefore examine whether institutions accept correction, explain their limitations and protect vulnerable people. Descriptions of a wise machine cannot excuse irresponsible human governance.

This also challenges churches. Using AI to save time may be reasonable. Allowing convenience to weaken confidentiality, personal attention or responsibility requires a different judgment. Religious language attached to a product does not remove the need to examine what its use does to a community.

The same discipline applies to predictions of extraordinary benefit or catastrophe. Such claims need evidence, assumptions and explicit uncertainty. Their resemblance to religious narratives may be worth analyzing, but resemblance alone neither confirms nor refutes the underlying risk.

Anthropic’s approach raises legitimate questions because moral language shapes expectations. Readers should neither mistake a company’s aspirations for demonstrated machine virtue nor dismiss every effort to build safer systems as religious pretension.

The immediate responsibility is concrete: require evidence for claims about AI, accountability from its makers and careful judgment from its users. Churches can contribute most by remaining thoughtful participants whose approval cannot be purchased with the vocabulary of wisdom.