Generative Artificial Intelligence: Policy Intersections, Considerations, and Recommendations | Page 8
and due to the stochastic nature of
generative AI, there is no guarantee one will
ever emerge.
There are steps educators can take to
better detect the use of uncited generative
AI content while continuing to provide a
supportive and welcoming learning
environment for students. These steps
involve engaging students directly in
discussions about the acceptable use of AI
and the importance of referencing sources
in their work. The triangulation of multiple
indicators of generative AI use is
recommended. Educators are encouraged
to consider and explore the following
approaches to generative AI detection
alongside any methods or approaches
already in place.
3.4.1 | AI Detection Software
Since the release of generative AI services
like ChatGPT, individual developers and
companies have rushed to create tools to
detect the outputs of generative AI. The
makers of these tools boast accuracy levels
of up to 99% when used to detect writing
created by artificial intelligence.
10
A
selection of these service providers include
TurnItIn, Copyleaks, GPTZero, and
Originality AI.
Performance testing of AI detection tools
demonstrates a propensity for false positive
outcomes across the industry.
11
12
AI
detection tools have also exhibited biased
performance when analyzing text written
by non-native English writers, with one
10
Desaire, H., Chua, A.E., Isom, M. et. al. (2023). Distinguishing
academic science writing from humans or ChatGPT with over 99%
accuracy using off-the-shelf machine learning tools. Cell Reports
Physical Science 4, 6. https://doi.org/10.1016/j.xcrp.2023.101426
11
Elkhatat, A.M., Elsaid, K. & Almeer, S. (2023). Evaluating the
efficacy of AI content detection tools in differentiating between
human and AI-generated text. Int J Educ Integr 19, 17.
https://doi.org/10.1007/s40979-023-00140-5
noteworthy study demonstrating a false
positive rate of 61.3%.
13
For these reasons,
educators should carefully consider use of
these tools in the analysis of student work.
The Minnesota State system office does not
recommend the use of AI detection tools as
singular indicators of plagiarism.
3.4.2 | Benchmark Writing Samples
Obtaining a benchmark writing sample from
a student can be a helpful comparative tool
when attempting to detect the use of
generative AI in academic writing. There are
a variety of ways to attain a benchmark
writing sample.
For example, one approach is to provide a
short essay question in the first weeks of a
course focused on a topic of great personal
interest to a student. This essay should
encourage a student to pull from their
personal experience and tap into their
interests, hobbies, and ambitions. This
essay question may reference course topics,
but it can also stand alone. If the class
meets in person, students can be asked to
write the essay while in class to add an
additional layer of ensured authenticity.
3.4.3 | Presence of Inaccuracies and/or
Inconsistencies
Generative AI services such as ChatGPT can
“hallucinate” in circumstances where the
user does not provide specific operating
instructions. These hallucinations typically
appear as outputs containing inaccurate
information or nonsense. The presence of
this information can be an indicator of
12
Sadasivan, V.S., Kumar, A., Balasubramanian, S., et. al. (2023).
Can AI-Generated Text be Reliably Detected?. Arxiv.org. Retrieved
November 22, 2023, from https://arxiv.org/pdf/2303.11156.pdf
13
Liang, W., Yuksekgonul, M., Mao, Y., et. al. (2023). GPT
detectors are biased against non-native English writers. Patterns
4, 7. https://doi.org/10.1016/j.patter.2023.100779