Can We Really Detect AI Writing in Research Papers? Testing SciSpace AI Detector

Welcome to Doctor Square. In this video I discuss one of the most urgent questions in today’s academic world: Can we really detect AI-generated writing, especially in scholarly texts? With the rapid rise of tools like ChatGPT, many teachers and researchers have noticed that students increasingly rely on generative AI. While AI offers powerful support for learning and productivity, it also raises serious concerns about academic integrity and ethical use. The real challenge is that, until now, we have not had a truly reliable way to distinguish between human-written and AI-generated academic writing. Most existing AI detectors are trained on general internet text such as blogs and websites. However, academic writing follows a very different structure, with formal tone, discipline-specific language, and complex argumentation. This is why many detectors fail when applied to research-style texts. In this video, I review a recent research paper that evaluates several AI detection tools and introduces the SciSpace AI Detector, a system trained specifically on research papers. According to the study, the detector was tested on over 4,000 passages and achieved an F1 score of 96.2% on scholarly writing, suggesting strong accuracy in distinguishing AI-generated text from human academic work. I explain what the F1 score means in simple terms and why it matters for evaluating detection performance. Then, we move from theory to practice. You will see a live demonstration where I: Test purely AI-generated text, Test AI text designed to sound more human, and Test excerpts from my own published academic writing. The results show that the SciSpace detector correctly identifies AI text as AI and human academic text as human, even when the AI is prompted to mimic a natural writing style. I also show how the tool allows users to paste text or upload full PDFs for analysis. However, I strongly emphasize that AI detectors should be used only as screening tools, not as final proof. No system is perfect. Any flagged case should always involve human review, transparency, and a fair appeal process. Technology can assist educators, but judgment must remain human. This video is especially useful for teachers, supervisors, researchers, and students who want to understand how AI detection works and how such tools can be used responsibly in academic settings. If you are dealing with essays, theses, or research papers, the SciSpace AI Detector appears to be one of the most promising tools designed specifically for scholarly writing. That’s it for today’s video. If you found this discussion helpful, please like, subscribe to Dr. Square, and share this video with your colleagues and students. Thank you for watching.