3.1. Course artefacts (adherence)
A requirement of the group written article was to include in the
appendices of the article a description of AI tools used. The
requirement was written as follows, Declaration of the databases
and tools used (including any AI tools), when applied to assist the
completion of the article, with a description of how they were used. If
AI tools have not been used, include a reflection explaining the
reasons. Further, items 2-4 of the learning agreement required students
to include specific information in their reporting of AI tool use.
Item 2: The appendix must (1) document what tool(s) were used, (2)
how the tools were used (i.e., prompts used), and (3) how the results
from the AI tools were incorporated into the submitted work.Item 3: Any content (text, image, data) from an AI tool must be
cited according to APA standards
(https://libguides.mcmaster.ca/cite-gen-ai/apa). This applies to in-text
citations and references (i.e. paraphrasing, quoting, or incorporating
any content from an AI tool into your own work).
Item 4: I am obliged to use AI tools in a responsible manner.
This means that as part of the appendix I submit with course
assignments, I must explicitly comment on actions taken to mitigate
and/or acknowledge the known limitations of the AI tools.
In the course a total of 17 group written articles were submitted. To
rate compliance, two raters reviewed the article appendices and rated
them for compliance using the following 4-point scale: ‘nothing’ (0)
with no text relating to the specific item, ‘incomplete’ (1) with text
relating to the specific item but addressing less than half of what was
requested, ‘partially complete’ (2) with text that addressed more than
half but not all of what was requested, and ‘complete’ (3) the text
fully addressed what was requested in the learning agreement item. In
total, 51 ratings were performed and there was high agreement with
initial ratings (86.27%). Disagreements were resolved through
discussion. Table 2 shows the results of group written article adherence
to the items in the learning agreement. Results suggest that students
adhered to the article requirement but not the learning agreement terms.
Although, it should be noted that a score of zero was assigned when no
specific text addressed the item in the student artefact. The
zero score does not distinguish the cases in which students did not feel
it was necessary to add specific text. For example, 7 of the 17 groups
declared that they did not use GenAI to support their work. Hence, it is
possible that they did not feel obliged to elaborate on Item 3
(Appropriate citations) and Item 4 (Responsible usage). Only 1 of the 10
groups that did declare use of GenAI provided prompts used via a link to
a ChatGPT thread, and none of these groups cited OpenAI in their text or
as part of their listed references. Also, only 1 of the 10 groups
explicitly acknowledged GenAI limitations, including biases in the
training data, the reliability of AI-generated content, and fairness in
AI-powered writing systems.