Full Report
I've been testing AI content detectors for two years now. They're getting more and more reliable.
Analysis Summary
# Main Topic
The increasing reliability and accuracy of AI content detectors when assessing text generated by Large Language Models (LLMs) like ChatGPT, observed over a two-year testing period spanning January 2023 to February 2025.
## Key Points
- **Improved Accuracy:** In early tests (Jan 2023), the best detector achieved 66% accuracy from three available checkers. In the most recent tests (Feb 2025), up to 10 checkers were used, with three achieving perfect scores (100%).
- **Plagiarism Context:** The use of AI-generated content without attribution meets the standard dictionary definition of plagiarism ("to steal and pass off... words of another as one's own").
- **Testing Methodology:** The reliability test involved feeding five text blocks (two human-written, three generated by ChatGPT) into various detectors and scoring them, with results above 70% treated as a strong probability verdict.
- **Detector Evolution:** One tool examined, ZeroGPT, transitioned from appearing "sketchy" (lacking clear monetization/contact) to presenting as a typical SaaS service, with its accuracy increasing from 80% to 100% across the evaluation set.
- **False Positives/Negatives:** The text raises concerns about whether human-written work is mistakenly flagged as AI-generated.
## Threat Actors
- **Threat Actors:** Not explicitly mentioned in the context of malicious activity. The focus is on the *producers* of content (users of LLMs) and the *defenders* (the detectors).
- **Attribution:** N/A.
## TTPs
- **TTPs:** The primary "technique" under scrutiny is **Content Generation via LLMs** (e.g., ChatGPT, Notion AI), which is then evaluated using **AI Detection Scoring**.
- **Adversarial Tactics:** Misrepresenting AI-generated content as original human work (falling under academic/editorial fraud/plagiarism).
## Affected Systems
- **Affected Systems:** Content evaluation platforms; specifically, the AI Detection market/tools themselves.
- **Scope of Impact:** Academic integrity, editorial integrity, and digital content review processes.
- **Specific Tools Evaluated:** BrandWell, Copyleaks, GPT-2 Output Detector, GPTZero, Grammarly, ZeroGPT, and Monica.
## Mitigations
- **Detector Utilization:** Employing modern AI content detectors, noting that some, like ZeroGPT, have shown perfect accuracy in recent tests.
- **Threshold Setting:** Establishing a strict threshold (e.g., >70% probability) when interpreting detector output.
- **Integrity Measures:** Implementing detection tools to safeguard academic or editorial integrity against uncredited AI content use.
## Conclusion
The threat landscape concerning unattributed AI content is rapidly evolving. AI detection tools have seen significant maturation and improved accuracy between 2023 and 2025, suggesting that tools capable of reliably identifying LLM-generated text are now available, offering a strong defense against plagiarism utilizing generative AI. Users are encouraged to test and deploy these solutions while remaining mindful of potential false positives.