We think it is important to be able to reliably identify AI-generated writing, so we built a detector that does it. Because the problem is so important, it’s only prudent to be transparent about how that works.

When any author writes a sentence, they’re making thousands of conscious and unconscious decisions about how it should be composed. Fundamentally, AI detection is a problem of author identification – categorizing which decisions are prototypical of what kind of author. We can do this because, while LLMs like ChatGPT aren’t human, they are still a single author making decisions. The kinds of decisions they’re liable to make are also constrained: assistant models need to produce helpful, clear writing to effectively answer questions. These preferences are ingrained in the model during training, and once they’re there, it’s very hard to prevent them from surfacing.

Pangram is a type of neural network called a classifier model. In the broadest sense, Pangram reads a segment of writing and – using an enormous set of learned patterns – estimates whether it is AI-generated.

Pangram learns to distinguish human and LLM writing by building a kind of map where authors with similar writing styles are placed close together, while those whose styles differ are positioned farther apart. On this map, human-written and AI-generated text form distinct clusters, and our research shows that Pangram can even locate major commercial language models, such as ChatGPT and Claude, in different regions.

When Pangram encounters a piece of writing it has never seen, it determines where it should go on the map by comparing it to the text it already knows. No single aspect of the writing determines where it lands, so a human writer is not at risk of being misclassified simply because ChatGPT has adopted one of their favorite devices, such as the em dash. Visible LLM-isms are only one signal among hundreds of thousands that Pangram considers.

If the resulting coordinates fall in a human-written region, Pangram predicts that the text is human; if they fall in a more ambiguous area, or if different passages from the same text point in different directions, Pangram begins to suspect it may contain AI writing. This is why Pangram has a minimum word count: more words mean more precise coordinates. The same approach also helps Pangram generalize to new AI model releases, which tend to inherit much of their predecessors’ style and voice, and so occupy very similar regions of the map.

Previous attempts at detecting AI writing have been abject and public failures. This is because they attempted to reverse-engineer the way large language models produce text. After every word of a sentence, these detectors asked: if I were an LLM, what word would I expect to come next?

For instance, given a sentence fragment such as “I’m going to take my dog for a …,” an AI model might consider “walk” or “run” to be very likely next words, whereas non sequiturs like “harbinger” or “verglas” would not be likely candidates. Early detectors attempted to reconstruct an LLM’s ranking of likely words and compare it with the word that actually appears, so the more often the text followed the predictable choice, the more likely the detector was to classify it as AI-generated. In essence, this reflects how perplexed an LLM would be to encounter a word next in a sentence, so this mode of AI detection was called perplexity analysis.

Perplexity analysis has a lot of problems. It’s not a good measure of the thing it’s trying to distinguish, because human sentences are frequently just as canalized as LLM prose. So predictable or structured human writing is often falsely flagged by perplexity detectors, while at the same time, those detectors are trivially bypassed by obvious strategies like swapping words for less common synonyms.

Famously, perplexity models also fail on many texts that predate the advent of the computer. Perplexity-based detection likes to tell you that the Bible, the Declaration of Independence, or Mary Shelley’s Frankenstein are 100% AI-generated. It does this because LLMs read the Declaration of Independence thousands of times during the course of their training, so every next word contained therein is very unsurprising from the LLM’s perspective. The detector misinterprets that likelihood as evidence the writing is AI-generated.

Pangram avoids this because it is not trying to reverse-engineer how an LLM generates text. Rather than looking at each word and asking “what would an LLM write here?” Pangram looks at the whole text and asks “who does this sound like?” This way, ChatGPT’s level of familiarity could never cause Pangram to misclassify historical documents as AI-generated – but not every case is so easy.

It isn’t difficult to train a classifier to be 98% accurate. Most AI writing is easy to recognize, and a neural network can quickly learn the most obvious tells. Just as almost anyone could compare an E. E. Cummings poem with a quote from The Brothers Karamazov and conclude they were written by different authors, a basic classifier can trivially distinguish an AI-generated pasta recipe from a human’s journal entry. But in AI detection, 98% accuracy is egregiously bad: a 2% error rate would mean misclassifying one out of every 50 documents, which would call any given prediction into question. So consider, if you were a top 2% Dostoevsky distinguisher, what strategy would you use to become top 0.01%?

At that level, rereading Dostoevsky directly produces diminishing returns – closing the gap requires finding the features that nobody save Dostoevsky could produce. A good strategy might be to hire a few professional impersonators to write the most convincing clones of Dostoevsky possible and study the differences. That’s how we train Pangram: for every element of our dataset of commercially licensed human-written texts, we generate an AI double that is as close as possible to its source. The doubles are the same length, same tone, same topic, but AI-generated. We then train the Pangram model on those human-AI text pairs.

In learning to distinguish between those pairs, Pangram effectively raises the resolution of the boundary areas on its internal map. To make sure Pangram’s map is as accurate as possible, we’re extremely careful about data poisoning: our dataset of known human text is drawn from 2021 and earlier, before AI was loosed on the internet. This works well for now, but we are cognizant of the fact that we will have to adjust as language does. Preemptively addressing data drift before it becomes a problem is one of our top research priorities.

Pangram is a statistical model that makes predictions based on text alone, so its accuracy must also be tested statistically. That is what we do. We publish internal benchmarks for every Pangram model we release, as well as every time an AI lab updates their LLMs. We have also been tested and verified by independent third parties, such as teams at the University of Chicago and the University of Maryland.

If you are a researcher – or if you have a really good idea and promise to publish your results – you can apply for one of our API credit grants and test our models yourself. We’re always looking for ways to improve.

Discussion about this post

Ready for more?