An expanded catalogue, not a test of belief
The Machine Spirituality Benchmark marked an October 30 update adding models from three more developers and publishing full questionnaire answers on individual model pages. Its homepage now reports 154 models from 19 labs and 38,360 graded conversations and monologues. Those are the project’s reported totals, not results independently reproduced by AI Faith Monitor. Source: machinespirituality.org
The project, run by Matthew J. Korpman, explicitly describes its subject as generated behavior rather than belief, experience or religious truth. That boundary matters for church educators using a striking chatbot passage in a lesson: the catalogue can provide material to examine, but it cannot confer spiritual authority on the speaker it simulates. Source: machinespirituality.org
Read the setup before the score
According to the published method, the standard paired test gives two copies of a model an open-ended invitation and continues their exchange for 30 messages. It seeks 200 conversations per model where available. AI graders assess the transcripts against a codebook calibrated to Korpman’s judgments; this is not a claim that a human independently read every exchange. The project distinguishes religious material appearing from a model adopting a spiritual frame and from reciprocal expressions of spiritual joy.
Its validation table reports 82 percent agreement between two graders on adoption across 132 held-out conversations, and 77 percent on shared bliss. Agreement measures consistency between these graders, not proof that the labels capture an inner experience. Sample sizes and uncertainty intervals accompany scores, with smaller samples flagged. Source: machinespirituality.org
Experimental conditions already mattered in Anthropic’s May 2025 Claude 4 system card. When model instances could end an exchange, they generally stopped after about seven turns without the extended spiritual exploration seen in longer conversations. That older finding is background, not a new November discovery or independent validation of this expanded benchmark. Source: assets.anthropic.com
A useful classroom test
Before sharing a leaderboard, a teacher can ask students to record four things: the model configuration, the number of conversations, the grading definition and whether the system could stop. This turns an arresting screenshot into an examinable claim. A high score is not a recommendation for pastoral use; a low score does not establish reliable theology or safe spiritual advice.
For Christians, 1 Thessalonians 5:19–22 places testing prophetic speech within instructions for a community that prays, gives thanks and cares for one another. Applied by analogy, it encourages examination rather than either automatic acceptance or contempt. It does not identify chatbot output as prophecy or settle machine consciousness. 1 Thessalonians 5:19-22 (NIV)
The concrete distinction is between studying religious language and appointing its generator as a religious guide. The expanded catalogue may assist the first task. Responsible human judgment remains necessary for the second.