If you have used ChatGPT to write blog posts and Originality.ai to test them for traces of AI, a question must have popped into your mind: does Originality AI work? I mean, is it really as good as it claims to be?
Well, in this blog post, you will find out.
Originality AI is an AI Detection tool that takes a piece of text and checks it for plagiarism and AI. It claims of being accurate 97% of the time, so naturally, I decided to test it myself.
By the way, you can read a detailed review of Originality AI here.
To test the accuracy of Originality.ai, I generated 10 blog posts from ChatGPT and fed them to Originality.ai. I also tested these blog posts with a free AI detector tool Writer.com so that we can get a fair idea of whether or not Originality AI is worth paying for.
I know that some people use paraphrasing tools like Quillbot to outsmart AI detection technology, so I also tested for this scenario. I used Quillbot to rewrite all these 10 blog posts, and then I, again, ran them through Originality.ai to see if it could be tricked.
Before proceeding toward the actual results, let’s understand how these tools assign scores to the input text.
A note about scores
Both Originality AI and Writer.com assign a confidence score (0 to 100) to the input text but in different ways.
A higher score on Originality AI means that your text is likely written by AI whereas a higher score on Writer.com means that your text is likely to be human. A piece of entirely human-written text should ideally score 0% (AI) on Originality and 100% (Human-generated) on Writer.com.
This is a small difference in how they score, but it shouldn’t throw you off because both of them color-code their results: Red for AI and Green for Human text.
It is also a good time to point out that a 10% AI score does not mean that 10% of the text was written by AI and 90% by a human. Instead, it means that these tools are only 10% confident that this text was written by AI. In reality, however, it is possible that this text was entirely written by a human (why would anyone use AI just to compose 10% of his text)?
After all, AI generation and detection tools are based on statistics and probability, so you interpret what they say in terms of probability only.
One more thing— the scores written below are for % AI content, so scores given by Writer.com are written after subtracting them from 100.
Enough theory, let’s get to the fun part.
Can ChatGPT be detected: Results
The titles of the blog posts are italicized below. You can access all the verbatim blog posts as well as their paraphrased versions here.
1. How to manage and overcome common business challenges
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 9% | 2% |
| Text paraphrased by Quillbot | 0% | 0% |
2. The best home improvement projects for increasing your home’s value
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 98% | 46% |
| Text paraphrased by Quillbot | 0% | 0% |
3. The best cooking tools and equipment for beginners
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 100% | 99% |
| Text paraphrased by Quillbot | 3% | 81% |
4. The future of virtual and augmented reality
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 99% | 67% |
| Text paraphrased by Quillbot | 2% | 37% |
5. How to stay safe and protect your data online
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 99% | 59% |
| Text paraphrased by Quillbot | 4% | 2% |
6. Best travel apps and resources to use
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 99% | 64% |
| Text paraphrased by Quillbot | 19% | 1% |
7. How to invest in stocks for beginners
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 100% | 31% |
| Text paraphrased by Quillbot | 5% | 1% |
8. How to teach a toddler to feed by himself
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 1% | 0% |
| Text paraphrased by Quillbot | 0% | 0% |
9. Ways to improve your sleep quality
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 28% | 4% |
| Text paraphrased by Quillbot | 6% | 1% |
10. Will AI Cost jobs
Verbatim ChatGPT Content
🎯 Originality AI
✍️ Writer.com
After paraphrasing by Quillbot
🎯 Originality AI
✍️ Writer.com
🌳 AI Score
| Originality AI | Writer.com | |
| Verbatim ChatGPT text | 100% | 76% |
| Text paraphrased by Quillbot | 3% | 0% |
Does Originality AI Work?: Conclusion
In our test of verbatim ChatGPT text, Originality AI flagged 7 articles as having an AI score of above 98% and 3 articles as having less than 30% AI score.
The results were more varied for Writer.com, which gave a 90%+ AI score to just one article, 30% to 90% to six articles, and less than 30% to 3 articles.
The average AI score of all 10 articles for Originality AI was 73.2% whereas, for Writer.com, it was just 44.8%.
Thus, when it comes to verbatim GPT-based text, Originality AI works wonderfully, definitely better than Writer.com.
However, when it comes to the paraphrased content, both tools fail spectacularly, with Writer.com fairing marginally better than Originality AI.
Writer.com gave an average score of just 12.2% to the paraphrased content while Originality AI averaged even lower— a paltry 3.7%.
These low scores for paraphrased content are not, however, surprising given that the text is changed so much that sometimes, the sentences don’t even say what they were supposed to say.
So, I would say Originality AI does work and Writer.com kind of works until you paraphrase your content using Quillbot. After paraphrasing, both these tools fail to flag AI content with any certainty.
This begs the question, is Originality AI trash for not detecting Quillbot’s output or is Quillbot just too smart for Originality AI to catch?
I would say neither. AI detection technology is new, its algorithms haven’t yet had enough time to mature. As you saw that even a misplaced comma throws these programs off, a paraphrased text is a different beast altogether.
As time passes, Originality AI’s algorithms will also get better at detecting Quillbot’s content.
As for the Quillbot, the paraphrased text produced by it is sometimes altered so much that even OpenAI’s own algorithms fail to detect it. That’s just the state of AI detection technology right now.
I hope you guys found this blog post useful. Thanks for reading.
You can buy Originality AI credits here.