AI detectors falling short in the battle against cheating

The statistics paint a grim picture. While the detectors exhibited near-perfection in evaluating essays written by US-born eighth-graders, they mistakenly categorized over half (61.22%) of the Test of English as a Foreign Language (TOEFL) essays written by non-native English students as AI-generated.

The situation worsens when examining the data a little closer. The study found that all seven AI detectors unanimously identified 18 out of 91 TOEFL student essays (19%) as AI-generated. An astonishing 89 out of 91 essays (97%) were flagged by at least one of the detectors.

Why the detectors falter in their accuracy 

Professor James Zou, a biomedical data science expert at Stanford University and the senior author of the study, explains the reason behind the detectors' unreliability. "Detectors typically evaluate content based on 'perplexity,' a metric that correlates with the sophistication of the writing," Zou states. 

"Naturally, non-native English speakers tend to trail behind their US-born counterparts in terms of linguistic complexity, resulting in lower perplexity scores."

Zou and his co-authors emphasize that non-native speakers often score lower on perplexity measures such as lexical richness, lexical diversity, syntactic complexity, and grammatical complexity.

These concerning numbers raise important questions about the objectivity of AI detectors. They highlight potential instances where foreign-born students and workers might face unfair accusations or even penalties due to false cheating claims. The ethical implications of such scenarios are a cause for concern.

Furthermore, Zou points out that the detectors can be easily undermined through a technique known as "prompt engineering."

By requesting generative AI systems to "rewrite" essays with more sophisticated language, students attempting to cheat can easily bypass the detectors. Zou provides a simple example, where a student utilizing ChatGPT might input AI-generated text with the prompt: "Elevate the provided text by employing literary language."

"Current detectors are clearly unreliable and easily manipulated, making them an unreliable solution to the AI cheating problem," Zou warns.

So, what is the way forward? Zou suggests a few possible approaches. In the short term, he recommends avoiding reliance on detectors in educational settings, particularly where there is a significant number of non-native English speakers. 

Developers should also move beyond perplexity as the primary metric and explore more sophisticated techniques. They should also consider inserting subtle clues about AI identity, like watermarks, into the generative content. Ultimately, models need to become less vulnerable to circumvention.

"At this time, the detectors are too unreliable, and the consequences for students are too high to put our faith in these technologies without rigorous evaluation and significant improvements," Zou concludes.