ILM prototype separates source approval from AI marking

What the prototype describes

An October 8 preprint by McMaster University researchers Suhaila Mohammed, Abdelaziz Serour and Allison Lahnala describes ILM, a prototype for learning Islamic prophetic narratives. It uses educator-approved stories and reviewed translations to generate learning activities.

Some questions use a structured story map and fixed answer matching. Open-ended answers are assessed by a language model against retrieved passages. A separately maintained Qur’anic layer is manually checked. The model’s grading instructions favour accepting an answer when the retrieved material is insufficient to establish an error.

The authors acknowledge possible extraction and question-generation errors. Work so far covers one story; formal studies with learners and educators are planned. This is an early design report, with educational effectiveness still to be established. Source: arxiv.org

What a correct mark should mean

AI Faith Monitor’s practical question is how an educator should interpret a positive mark. A child might correctly name a person in a narrative while misunderstanding the relationship between two events. Another might understand the passage but express the answer differently from a teacher’s example. These are hypothetical teaching situations, not findings about ILM.

Before adopting automated feedback, a religious educator should specify what the activity is meant to assess: recalling a detail, explaining a sequence or offering an interpretation. Each calls for a different kind of judgment. A mark can only be useful when its meaning is clear to the teacher and learner. An encouraging response should also leave room for a teacher to identify what still needs explanation.

An independent educational standard

UNESCO’s Guidance for generative AI in education and research, published September 7, 2023, calls for human-centred, age-appropriate ethical validation and pedagogical design, alongside privacy protection. Its official overview also identifies institutions’ capacity to validate educational tools as a concern. This is general education guidance, not an assessment or endorsement of ILM. Source: unesco.org

For a mosque, church or religious school, that places a practical responsibility with the teaching team: decide what evidence would justify using a tool for the particular learners and lesson in view.

Make disagreement reviewable

A useful first review could use fictional, adult-written answers rather than children’s records. Include a clear answer, a reasonable paraphrase, a partly correct explanation and an answer the lesson does not settle. Ask a qualified teacher to assess them first and explain the reasons. Any later software comparison should preserve disagreements for examination rather than reduce them immediately to an average score.

This is our proposed review exercise, not a validated intervention. It can help frame questions for a developer; it cannot establish improved learning. Before use affects a learner’s progress, educators would also need a workable route to challenge a mark, correct an error and explain the correction. The person responsible for teaching should remain identifiable when the feedback arrives through software.

A Christian teaching responsibility

For Christian educators, Matthew 13:51–52 places a question about understanding within Jesus’ teaching of his disciples. Our application is that attention to understanding belongs within teaching itself; an automated mark cannot relieve a teacher of that responsibility. This is a Christian educational reflection, not a judgment on Islamic teaching or evidence about software performance. Matthew 13:51-52 (NIV)