You’re Already Talking To AI, You Just Can’t Tell The Difference

Chatting With a Stranger Online? There’s a Good Chance It Isn’t a Person.

Scientists ran the original, stricter version of a famous test of machine intelligence, and the results should make anyone pause before assuming they know who’s on the other end of a chat window.

In a study published in Proceedings of the National Academy of Sciences, researchers put several AI systems head-to-head against real people in a classic challenge called the Turing test, a 75-year-old idea that asks whether a machine can hold a conversation convincingly enough that a human judge can’t tell it apart from another person. Under the right conditions, the answer is yes. Not just barely. In some cases, people were more likely to mistake the AI for a human than to correctly identify the actual person on the other side of the conversation.

That’s not a glitch in the experiment. That’s the point. When a specific AI system called GPT-4.5 was given detailed instructions on how to come across as a real person (a setup the researchers called GPT-4.5-PERSONA), human judges picked it as the human 73% of the time and were actually less likely to correctly identify the real person in the room. For anyone who has assumed they could just “feel” when they’re talking to a bot, this study offers a brisk correction.

How the Turing Test Actually Works

Most people have a vague idea of what the Turing test is: a machine tries to convince a human it’s real. But the version used in this study is more demanding than pop culture suggests. This is the three-party format, in which the human judge chatted with two strangers at the same time, one real person and one AI, and had to figure out which was which.

This setup is harder for AI to pass because the judge isn’t just asking “could this be a person?” They’re directly comparing two conversations. In older versions of the test with earlier AI systems, machines were correctly identified as non-human more than 90% of the time. Today’s AI can beat those odds, and in some cases beat the humans they’re competing against.

Why AI Fooled So Many People

Two groups participated in the initial experiments: undergraduate students at UC San Diego and adults recruited through an online research platform. After exclusions, researchers analyzed over 1,000 games, each consisting of five-minute text conversations.

Researchers tested the AI systems with two different sets of instructions. One gave only bare-bones guidance. The other, the PERSONA prompt, told the AI to act like a young, somewhat introverted person familiar with internet culture and slang. That difference mattered enormously. Without persona instructions, GPT-4.5’s success rate dropped to 36%, meaning judges were more likely than not to correctly identify it as the machine. A specific personality to imitate flipped the results entirely.

One of the more revealing findings is how people tried to spot the AI. Most assumed they’d catch it by asking something clever. Instead, the most common approach was small talk. In 61% of games, judges asked witnesses about their daily lives and personal details. About half the time, they probed for emotional warmth, asking about feelings, humor, and opinions. Only 12% asked logic puzzles or knowledge questions traditionally associated with testing a machine.

That number is telling. After 75 years of computers beating humans at chess and calculating in milliseconds, people no longer seem to think raw intelligence is what separates humans from machines. They’re looking for something warmer: personality, spontaneity, the feeling that someone genuinely gets the joke. Reasons judges gave for their verdicts focused heavily on writing style, conversational flow, and gut feeling. One participant admitted to picking a witness “because they had human vibes.” When judges were accurate, it was often because a witness dodged a direct question, or because they didn’t know something, which felt more human than an AI’s typically thorough answers.

Does a Longer Conversation Help? Not Really.

To address the concern that five minutes wasn’t enough time, the researchers ran a follow-up extending conversations to 15 minutes and preventing judges from ending early. In that study only, GPT-5 replaced GPT-4.5 because GPT-4.5 had been deprecated by OpenAI. Both GPT-5 and LLaMa-3.1-405B still passed. Judges remained unable to reliably tell them apart from humans, and none of the demographic factors researchers tracked, including age, education, chatbot use, and self-reported AI knowledge, predicted who would be more accurate.

What This Means for Anyone Who Chats Online

Passing the Turing test doesn’t settle every philosophical debate about whether machines are truly “thinking.” But what the study shows, in concrete numbers across multiple experiments, is that AI systems can already stand in for a real person in a short text conversation, and most people won’t notice.

Researchers use the phrase “counterfeit people” to describe systems capable of this kind of imitation, noting that just as fake currency erodes the value of real money, simulated human interactions could gradually erode the value of real ones.

Ultimately, the most unsettling detail isn’t that AI passed a famous test. It’s that the real human in the room, an actual person trying to seem like themselves, lost.

---

Disclaimer: This article is based on a peer-reviewed research study, but it is intended for general informational purposes only. The findings reflect results from controlled experimental conditions and may not apply to all real-world scenarios or AI systems.

---