A systematic evaluation of current AI text detectors has concluded they are "not trustworthy enough" to determine whether a student used AI, according to a new study highlighted by AI researcher Rohan Paul. The research, which appears to be published in a ScienceDirect journal, represents one of the most comprehensive real-world tests of detection tools to date.
What the Study Tested
The research team constructed three large datasets from authentic student work created before the generative AI era. This pre-GenAI baseline is crucial—it represents genuine human writing unaffected by contemporary AI assistance. For each dataset, the researchers created paired AI-generated versions using modern language models, creating a controlled comparison between human and machine text.
The team then ran these 280,000+ text samples through 13 different detection tools, including both commercial products and open-source solutions. The datasets covered different academic contexts: long-form theses, short coursework assignments, and engineering code submissions.
Key Findings: Where Detectors Break Down
The results revealed significant, context-dependent failures:
1. Performance Variation by Length
Detectors showed somewhat better accuracy on long-form theses but "broke down" on short coursework assignments. This suggests current detection algorithms may rely on statistical patterns that emerge only in longer texts, making them particularly unreliable for the brief assignments that dominate many courses.
2. Critical Failure on STEM Writing
The most concerning finding involves STEM (science, technology, engineering, and mathematics) writing. The study found that technical writing was "more likely to be flagged unfairly" as AI-generated. The researchers attribute this to the formulaic, precise nature of technical academic writing, which detection algorithms apparently mistake for AI-generated text.
3. Engineering Code Detection Problems
The study specifically notes detectors performed poorly on engineering code submissions, though the source doesn't specify whether this refers to code comments, documentation, or the code itself.
The Real-World Implications
These findings have immediate practical consequences for educational institutions. Many schools have implemented or considered AI detection systems to maintain academic integrity in the ChatGPT era. This study suggests such systems may be fundamentally flawed for assessing short assignments—exactly the type of work where AI assistance would be most tempting—and may systematically disadvantage STEM students whose writing follows established disciplinary conventions.
The unfair flagging of STEM writing raises particular concerns about equity and bias in automated assessment systems. If technical students are more likely to face false accusations of AI use, it could create disproportionate administrative burdens and potentially damage student-instructor relationships.
Methodology and Limitations
While the source provides limited methodological details, the approach of using pre-GenAI student work as a clean human baseline is methodologically sound. By comparing this authentic human writing to AI-generated versions of similar content, the researchers created a controlled test environment that avoids the contamination issues present in many earlier detection studies.
The study's scale—280,000+ samples across 13 detectors—provides statistical power that smaller evaluations lack. However, without access to the full paper, we cannot assess specific accuracy metrics, false positive rates, or which detectors performed best.
gentic.news Analysis
This study arrives at a critical moment in the AI detection arms race. As we reported in our coverage of OpenAI's discontinued AI classifier in July 2023, the company itself acknowledged its tool was "not fully reliable" and had a 26% false positive rate for identifying AI-written text. The current findings suggest the fundamental challenges OpenAI identified—particularly with short texts and specialized writing—remain unresolved across the detection ecosystem.
The STEM writing bias finding connects to a broader pattern we've observed in AI evaluation systems. In November 2023, we covered research showing that AI grading systems often penalize non-native English speakers and writers from certain cultural backgrounds. The current study extends this concern to disciplinary writing styles, revealing another dimension of algorithmic bias in educational technology.
This research also contextualizes the recent trend of detection tool withdrawals and policy changes. Following Turnitin's AI detection feature launch in April 2023, many institutions reported high false positive rates, particularly for ESL students. The current study provides systematic evidence for why these tools struggle, especially with the short-form writing that comprises most undergraduate coursework.
For practitioners, the takeaway is clear: current detection tools should not be used as sole arbiters of academic integrity decisions. The combination of length dependence and disciplinary bias creates unacceptable risks of false accusations. Institutions would be better served by focusing on pedagogical redesign—creating assignments that integrate AI thoughtfully—rather than attempting to police AI use with unreliable detection systems.
Frequently Asked Questions
How accurate are AI detectors for student work?
According to this study of 280,000+ samples, AI detectors perform unreliably, especially on short coursework assignments and STEM writing. While they show somewhat better performance on long-form theses, they frequently flag authentic student work—particularly technical writing—as AI-generated due to its formulaic nature. The researchers conclude current detectors are "not trustworthy enough" for making determinations about student AI use.
Why do AI detectors fail on STEM writing?
The study found STEM writing is "more likely to be flagged unfairly" as AI-generated because technical academic writing often follows established conventions and precise language patterns that detection algorithms mistake for AI-generated text. The formulaic nature of scientific writing—with its standardized structures, terminology, and objective tone—apparently overlaps significantly with how current language models generate technical content.
What types of student work are hardest for AI detectors to evaluate?
The research identified three particularly challenging contexts: (1) short coursework assignments (as opposed to longer theses), (2) STEM and technical writing across disciplines, and (3) engineering code submissions. Detectors showed the poorest performance in these areas, suggesting they may be fundamentally unsuited for evaluating the types of assignments most common in undergraduate education.
Should schools use AI detectors for academic integrity?
Based on this research, current AI detection tools should not be used as the primary or sole method for determining academic integrity violations. The high risk of false positives—especially for STEM students and short assignments—creates significant fairness concerns. Educational institutions should consider alternative approaches, including assignment redesign, process-oriented assessments, and educational conversations about appropriate AI use in learning contexts.
