Website 'Human or Not?' uses AI to create a top-tier Turing Test

Thanks to “Human or Not?”, a website launched on Wednesday by leading Israeli generative artificial intelligence developer AI21 Labs, you can find out for yourself — but you may be frustrated by the results.

In an attempt to learn more about the nature and capabilities of AI bots in 2023, AI21 Labs’ new website acts as one of the most advanced Turing Tests ever created, putting users into a two-minute anonymous chat with another user — or with an AI model pretending to be one. At the end of the brief conversation, users are tasked with determining whether the other chatter was indeed human.

The game features a stable of the world’s top-tier learning language models including OpenAI's GPT-4, AI21's Jurassic-2, Anthropic’s Claude and several others, all of which have been configured to mimic human typing and conversation — including slang, typos and mistakes. Because of this, it can actually be remarkably difficult to determine just what is on the other side of the screen.

Incorporating AI into our lives

Despite its fun premise, the ultimate goal of the app isn’t to be a fun game — it’s to start a discussion among people and policy-makers regarding the incorporation of AI into our lives, according to the project’s leader, Amos Meron.

“There's a few levels of discussion that we hope to see come from this experience,” Meron said. “The first level is obviously the people: people are going to start thinking about it themselves. That's the most important. Then I think there's the higher level of policymakers and [the media], asking ‘What does this mean? What should we do next? Because there's certain things that people feel comfortable with an AI doing for them, and certain things not.” The hope is that “Human or Not?” will further the efforts in discovering that fine line, he explained.

Though AI21 has yet to release any specific statistics, the site appears to have garnered significant attention. “Right now we're in the very early beginning of the game, so I don't want to commit to any numbers, but I’ll just say that, given the fact that the game has been alive for just about 24 hours, we’re seeing very nice numbers. People are very engaged,” Meron said.

In fact, users are so engaged that they keep coming back for more, which came as a pleasant surprise to the team at AI21. “It's just fun to see that people keep coming back to play like, five or ten games,” said Meron. “It's fun for people, but I also think it's very interesting that people are not bored after one game or two. We’re seeing that people are coming up with very different strategies.”

AI21 Labs was founded by Stanford professor emeritus Yoav Shoham, Ori Goshen (founder of CrowdX) and Prof. Amnon Shashua (founder of Mobileye). The team has built one of the largest and most advanced language models in the world, Jurassic-2, which is now also a part of the recently launched Amazon Bedrock.