Does Undetectable AI work?

AI detectors can spot AI-generated content.  But do they work? Undetectable AI is a startup company and software that's gained notoriety for its adversarial approach to detecting AI-generated content. 

It claims to detect and modify AI-generated content so that AI detection programs can't spot it. I decided to test undetectable AI. 

First, I went to ChatGPT 4.0 and asked it to generate an essay about climate change.

Next, I took the essay that ChatGPT gave to me and ran it through undetectable AI's AI detector. The Undetectable AI detector successfully identified the ChatGPT4o content as AI-generated.

(screenshot: Undetectable.ai)

I also put the AI-generated text through ZeroGPT, and the results returned as AI-generated.

Is it possible to fool AI detectors?

Over the last year, there's been controversy surrounding AI detectors and AI detection technology, specifically regarding people's essays getting flagged by AI detection systems like Turnitin or GPTzero when they claim they never used AI. 

Bloomberg reported on this phenomenon last month, and it's been a complaint in various news articles over the last year.

Testing Undetectable AI’s Adversarial Features

I took the same chat GPT-4-0 text that Undetectable and ZeroGPT detected as AI and ran it through Undetectable's AI humanizer, which claims to make the content indistinguishable from human-written text.

After getting the humanized text back from Undetectable AI, I put the content through ZeroGPT's AI detector to see the results of the ‘humanized’ text.

According to the ZeroGPT AI detector, the humanized AI text scored as human. Reading the text AI humanizer generated sounded more natural and less rigid. 

A Cause For Concern?

AI detectors work sometimes. That is to say that they can accurately detect AI-generated text. However, when adversarial strategies are used (like undetectable AI), it's clear that AI-detectors can be fooled. 

While it's interesting to see companies developing AI technology that’s becoming indistinguishable from human output, some might find it a bit concerning. For instance, one worry people have is that AI could replace humans. However, another implication of undetectable AI technology is the ability for people to abuse it.

For example, if I was a student who wanted to have ChatGPT generate my essay, but I didn't want anyone to know about it, I could theoretically use undetectable AI to ensure that I don't get caught.

It's worth noting, though, that on the bottom left-hand side of the undetectable AI text box,  there's a button you have to click that says you agree to the terms of service, which includes no academic misconduct.  And undetectable AI publicly stated that they don't condone cheating in the past. But to what degree they enforce that, we don't know.

There are positive use cases for such a technology, though. One example is that those prone to having their writing style flagged as AI can use Undetectable to avoid being penalized unfairly.  

In other cases where disclosing one's use of AI isn't necessary, software like  Undetectable could help ensure your content comes across as authentic and not AI-generated.

The Verdict

From the tests I ran using the Undetectable AI detector and the Undetectable AI humanizer, I found that they both worked. 

The first text that I checked on Undetectable AI, I also checked on zero GPT, and both marked the text as being AI-generated correctly.

To test out undetectable software, I had to contact customer service to set up an account for this article. When I reached out, they quickly responded to me. So that's a big bonus because they have 24-hour live customer service, which most companies don't have these days.

I also tested the undetectable AI humanizer, which also worked and delivered on its promise to make the text appear more human.

My only complaint is that it seems Undetectable AI doesn't enforce content moderation policies.  For example, there doesn't seem to be a process beyond checking a box to prevent students from cheating.

Undetectable AI works, but users should be careful when using it so as not to violate any ethical or legal policies. It’s also worth noting that no AI tool is 100% accurate, and when using one, you should use caution and sound judgment.

This article was written in cooperation with Tom White