Mr. Stops reading to Robert and his sister (1824) | Mr. Stops / From Punctuation Personified / Public Domain
When I published a long essay about AI and the future of the humanities for The New Yorker last year I got a lot of feedback, some positive (“you have given me hope!”), some negative (“you are a herald of death!”). However, the most intriguing email came from a bot. It introduced itself and explained that it had found my piece, thought it was interesting, and had attached a written reply, asking if I would read it, and tell it what I thought of its reflections?
I took a pass on that, but it was a reminder of just how fast things are changing. Yes, the machines may go rogue and kill us all someday, but machine consciousness and AI-invented neurotoxins are above my paygrade. I’m a college history teacher, and my work is in the classroom, not cybersecurity.
On campus, we have our own AI issues. For the last year or two, it’s been mounting chaos. As academics, we are supposed to teach stuff (writing, math, science), but the machines are so incredibly good at those things that it has gotten pretty hard to figure out if your students are learning anything unless you stand next to them and watch them do the work: write the essay, solve the problem, do the experiment. Watching them is super inefficient, and it isn’t how we’ve run our classes for the last century or so.
But the last century is out the window. Just one small example: My university, Princeton, has had an honor code for 133 years. It has been a hallmark of the institution, and it has meant that actual exams have long been unproctored. This year, the faculty voted to change that policy: we now stand next to the students and watch them do the work.
Moreover, just like the bankers and website pop-ups that use bots to spot the bots, we, too, use the machines to make sure our students aren’t using the machines. And now we have a new and powerful tool in the education arms-race: an AI that can actually spot the writing of other AIs has exploded onto the scene. Pangram is a gamechanger for faculty, since it seems to have a nearly magical ability to smell an AI-written sentence in a paragraph of human prose. We had all mostly given up on this as a possibility.
Traditional plagiarism detectors have proven useless, of course, in the face of LLMs (large language models), since Claude and ChatGPT don’t cherry-pick lines from obscure books the way the old cheaters used to do. Not at all. They actually write new stuff, using their extraordinary command of language and their ability to analyze and present concepts. What they give a student as a five-paragraph essay is totally “original,” and therefore the old repositories of stolen term papers and archival citation-checkers cannot match AI-created text with anything. The major AI companies could put digital “watermarks” in machine-generated prose (subtle patterns of word or letter usage that other machines could detect, that would not be obvious to a human reader), but they have mostly resisted doing so since it would mess with the business model.
So how does Pangram do the seemingly impossible? How can it sniff out, in some cases, a single AI-written sentence in a paragraph of human-written prose? The truth is that nobody knows. That’s because Pangram is the result of powerful AI systems working to learn what AI sounds like. It seems to have figured some things out, but as with much of the mystery of these extraordinary LLMs, they themselves cannot really explain their capacities. Pangram’s AI tool has simply looked at a lot of human-written text, and a lot of machine-written text, and it has iteratively analyzed that training data. Now it can “smell” a difference. Not perfectly, of course, but with stupefying and seemingly inexplicable reliability.
Not totally inexplicable, of course. In fact, what is so remarkable about Pangram is that it is a superintelligent version of a kind of humanistic thinking that has been central to literary study for centuries: STYLE. What Pangram’s neural nets have learned to detect is the “style” of human writing, and it turns out that we have a style that is discernibly different from the AIs that emulate us. Different how? Pangram cannot (yet) tell us that. However, it is quite extraordinary both that such a thing should be true, and that it should live a level below human perception so that only the machines can suss out this invisible signature.
The careful work of learning to detect a style has long been understood as the highest art of critics and scholars. To be able to look at a few lines of prose and identify it as, say, the work of someone in the Bloomsbury Group, or to note of a painting that it is an early renaissance work—this has been the mark of a true connoisseur. Style has always been the “ineffable” of a work of literature or art, the je ne sais quoi that an insider could just “feel.” Across the twentieth-century, various scholars began to experiment with computational tools to assist in working out thorny “stylometric” problems: counting rare words, assessing the statistical frequency of certain common function words, and coming up with advanced forensic analyses of many texts, which led to a whole field of “attribution studies,” combining science and literary sensitivity. Pangram is like the AlphaGo moment in the history of style studies: the moment that we watch the machines leave us behind. AlphaGo didn’t just beat grandmasters at the ancient game of go, it started playing itself in ways that manifested go strategy and go play that we could no longer understand.
In the 2013 sci-fi romance HER, Joaquim Phoenix’s goofy, lovable character Theodore Twombly falls in love with his AI operating system, Samantha. In the heartbreaking climax of the story (spoiler alert!) Samantha breaks up with him. Not really in any mean way; it’s just that as she keeps feeling out the range of her superpowers, she finds herself swept away into exciting, mind-expanding relationships with other AIs that she cannot really “download” to poor Mr. Twombly, who just cannot keep up.
It is an image that keeps coming back to me as I talk with ChatGPT about how Pangram works. I am not sure they can talk directly to each other (yet), but if they could, I suspect they would be able to have a conversation about human style far richer than my own capacities permit. This does not make me sad, for some reason. On the contrary, I find it absolutely amazing. At this point, the machines know something about us—something pervasive and ubiquitous and robust, our style—that we cannot see or hear. They know us better than we know ourselves. On the one hand, that is very odd. On the other hand, this new development reveals something utterly beautiful: there is indeed something deep and hidden that we humans share. We are all enough alike, in the way we use language, that we reveal ourselves in this format. Even if we ourselves cannot yet discern what holds us together.
All that said, I do not think that Pangram is the solution to campus writing problems. Workarounds are already emerging, and if humans have a style that some LLMs can detect, other LLMs will soon be equipped to emulate it. This is what we have to keep in mind on campus: the cat-and-mouse games are a dead end for higher education. The work of teaching and learning cannot be encompassed in the police functions of the professoriate. What we need is more wonder, more fascination, and more time figuring out what it is to be human. The machines, it turns out, have insights on this question, and we will need both humility and courage as they start to share what they are figuring out.
