In this article, you will learn how to detect ChatGPT text with the help of 6 AI detection tools.
Detecting ChatGPT is important for universities, professors, and for content marketers who outsource their content writing. You must know if you are getting ripped off.
In this article, I will first generate a few paragraphs using ChatGPT and then feed it into 5 free and 1 paid ChatGPT detectors and see how they perform.
Before moving on to our case study, let’s first understand how ChatGPT detectors work.
How ChatGPT detectors work
ChatGPT detection tools rely on the same technology that ChatGPT itself uses to generate text: Large Language Models (LLMs).
GPT-3 and GPT-4 are the largest and most famous LLMs but there are others too such as Bloom, T-5, LaMDA (from Google), RoBERTa, etc.
ChatGPT uses GPT-3 and GPT-4 to produce text using some formulas of probability and statistics. It computes the probability of what word can come next, based on what words have already come before. The word with the highest probability wins.
ChatGPT detectors use the same LLMs (and the same formulas or probabilities) to find out if a given text was written using ChatGPT.
Think of it like this: if you write an essay from the same book that your teacher has also read, you will most likely get caught.
It also means that an AI detection tool that is trained on one LLM may not detect text that was produced using any other LLM. For example, OpenAI’s AI Text Classifier is trained on GPT-based models, so it may not successfully flag content written using LaMDA or RoBERTa.
Limitations of ChatGPT detectors
No AI detection tool is perfect yet. There are a few limitations that you must know about:
Easy to fool
AI detection tools are pretty easy to fool. Even a slight change of comma or spelling or a change of word confuses them.
I conducted an experiment with Writer.com’s AI detector where fresh ChatGPT text scored just 26% human-written. Placing a comma somewhere increased this human score to 69% and putting another comma pushed it up to 99% human-generated.
It is for this reason that many people use paraphrasing tools like Quillbot to evade AI detection.
False positives and False negatives
ChatGPT detection tools are not yet accurate enough. They sometimes flag ChatGPT-produced text as human-written (false negative) and other times, flag human-written text as AI-generated (false positive).
Limited by the model they are trained on
As we already discussed, AI detection tools can only detect text written by a model that they are trained on.
So, if someone writes a text using a language model other than GPT-based, that text may not be correctly flagged.
Work best with longer input
Since AI detection tools work by computing the probability of words coming next based on the words that have come before, they tend to work better with longer input.
An input of fewer than 100 words isn’t sufficient enough to gauge the pattern of words chained together by an AI. Shorter input pushes up the chances of AI detectors producing false negatives and false positives.
So, aim to provide input of more than 100 or 150 words for more reliable results.
We are done with the limitations. Let’s now learn how to detect ChatGPT text.
To demo the tools listed in this article, let’s first generate some content using ChatGPT.
I asked ChatGPT to write a 200-word paragraph on “How to reduce belly fat”. Here’s its response:
Neat!
Let’s now copy this text and try these tools to detect ChatGPT text.
6 Tools to Detect ChatGPT Text
There are free as well as paid tools on the market for detecting ChatGPT text.
The free ones, understandably come with some limitations, such as a maximum number of characters that can be scanned at a time, no highlighting of suspicious text, no saved history, no plagiarism check, and no Chrome extension.
If these things are important to you, a paid tool like Originality.ai (demoed at the end of this blog post) will be suitable for you.
A note about scores
AI detection tools usually assign an AI score to the input text, that ranges from 0 to 100. This score represents the probability of content being written using AI and not the percentage of content that was written using AI.
For example, a score of 85% AI means that the tool is about 85% sure that AI wrote that piece of text and not that 85% of the text was written using AI.
Okay, enough theory. Let’s start with the free tools first.
1. ContentDetector.ai
The first tool on this list is ContentDetector.ai. It claims to be free, and there seems to be no limit on the maximum number of characters/words you can test at a time.
To test this tool, I generated 2 sample paragraphs about Bitcoin with the help of ChatGPT and as expected, it correctly identified that text as AI-generated (probability > 90%).
ContentDetector.ai also appears to be immune to the paraphrasing tools like QuillBot. I paraphrased the above text with 6 different modes of QuillBot and every time, it was able to flag the text as “likely AI-generated”.
However, when I introduced a few intentional grammatical and spelling errors in my sample text, its AI-detection flag dropped from 90% AI to 65% AI. But not much blame can be put on ContentDetector.ai because all tools in the market suffer from this issue right now.
Even a tiny mistake in your input text sometimes trips AI-detectors and makes them lose their accuracy drastically.
Okay, let’s move on to the next free tool to detect AI text.
2. ZeroGPT
ZeroGPT is relatively new in this game, its website is just 4 months old, at the time of writing of this article. It’s currently being offered for free.
It claims to have analyzed 10 million articles and achieved a success rate upwards of 98%. It uses a proprietary technology called DeepAnalyse™ for identifying the origin of the given text.
Let’s put it through its paces:
Yup, it succeeds. It correctly flags our content to be AI/GPT generated.
Great!
3. Sapling.ai
Sapling.ai is another free tool that outputs the probability that a given piece of text was written using GPT 3.5 or ChatGPT. Its free version supports testing for about 2000 characters and paid version supports checking 8000 characters at a time.
Let’s take it for a ride:
It correctly flags this content to be fake or AI-generated.
In my testing, I found out that this tool was too unwilling to label any text as human-written, to the point of being useless. Frequently, it identifies truly human-written text as 100% Fake (AI).
For example, I took some text from this article on indeed.com, a reputed website, and fed that text to Sapling.ai. To my utter surprise, it labeled it as 100% fake, even though it couldn’t highlight any sentence that it thought was AI written.
In contrast, all other AI-detection tools on this list identified that text as produced by a human.
I would stay away from this tool for any serious business.
4. Writefull
Writefull is another great free tool that can check your text for traces of GPT-3, GPT-4, or ChatGPT.
Let’s test it out:
It is only 74% sure that the text was written using ChatGPT.
Yikes! Not that great.
5. Writer.com
Writer.com is primarily a one-stop hub where writers hang out. It offers a paid AI writing tool and free AI detection software.
Its AI detection tool currently supports a maximum of 1500 characters. It claims to flag text that was written using GPT-3.
Let’s test it out:
It identifies it as only a 36% probability of human-generated content, which means a 64% probability of AI content.
6. Originality AI
Originality AI is a paid tool, that makes tall claims of detecting GPT-3, GPT-3.5, and GPT-4 content.
Read Originality AI’s detailed review here.
Let’s test if those claims are true or not.
It correctly detects this text as 100% AI-generated.
Some people nowadays rely on paraphrasing tools like Quillbot to rewrite the AI content to evade AI detection. Originality AI claims to catch those paraphrased versions too, with more than 96% probability.
Let’s paraphrase this text using Quillbot and test it again with Originality.ai.
Now, let’s copy this paraphrased version and feed it to Originality.ai.
Yup! It again detects the paraphrased version as 100% AI. That’s impressive.
If your workflow requires you to frequently use AI detection, I would recommend going with Originality AI. It comes with a browser extension, text highlighting, plagiarism check, saved history feature, and the capability to add team members.
It’s pretty cheap too (costs just $0.01 for scanning 100 words). It’s not a subscription, you only pay for what you use.
Final thoughts
In this blog post, we saw how you can detect ChatGPT text for finding out if the content has been written using AI.
Remember, no one tool is perfect. They are susceptible to their flaws, the most prominent of which are false positives and false negatives.
If you want a reliable free AI detector, Writer.com or Writefull may serve your needs but if you need advanced features such as saving the text, sharing between teams, a plagiarism detector, and a ubiquitous browser extension, you should go for Originality.ai.
It’s ridiculously cheap ($0.01 for 100 words) and excels at what it does.
Thanks for reading.
Final Reading: