Ghost writer in the machine

Editor’s note: An AI detector tool Pangram has become the tool for frenzied witch-hunts to expose writers who secretly use AI. Among the accused, everyone from literary prize winners to prominent economists. Founding Editor Lakshmi Chaudhry explains why this latest version of the Salem trials is absurd and perhaps dangerously distracting.

Written by: Lakshmi Chaudhry

*****

The very first internal combustion engine was invented in 1885. Then came the first recognizably modern car, the 1901 Mercedes. By 1915, despite great resistance from the horse and carriage industry, clunky, unreliable automobiles carried the day because of… shit, specifically horse shit:

By the 1890s, about 300,000 horses were working on the streets of London, and more than 150,000 in New York City. Each of these horses produced an average of 10kg of manure a day, plus about a litre of urine… A newspaper editor in New York City said in 1857 that “with the exception of a very few thoroughfares, all the streets are one mass of reeking, disgusting filth, which in some places is piled to such a height as to render them almost impassable to vehicles”.

But here’s the real kicker: The Great Horse Shit Problem was itself sparked by an earlier technological invention: the steam engine and the network of railways that allowed the mass and rapid movement of goods and people between towns and cities. Goods and people that needed to be transported within these cities—which sparked an exponential demand for horses, resulting in vast amounts of shit. To sum up: A great technological leap created an unanticipated side effect, which in turn was solved by another invention, the automobile. 

The panic over AI, hence, feels comfortingly familiar to its supporters. Luddites whinge, but technology prevails. More things change etc. Except for one small and significant break from the past: this time, the revolutionary new tech is itself churning out vast mountains of manure.  

Much of this is a ‘garbage in, garbage out’ problem. Humans have spent much of the past two decades turning the internet into a giant landfill (See clickbait, ragebait, engagementbait, linkbait etc etc). All of it swallowed whole by ‘training’ machines. Almost five years into the modern generative AI revolution, we are faced with a strange conundrum: Humans rely on AI-generated manure to put out more manure—which in turn is regurgitated by the machines to… You can see where this is going. The great fear is that the means of production of knowledge has turned into a giant recycler of shit.

***

There are two parts to this great fear. First is that machines will destroy human creativity. Hence, the hysteria over undereducated students who use ChatGPT to skate through college. Kids are not all right because they are outsourcing comprehension and analytical skills to a machine—a not very bright one at that. Hence, the outraged calls to ban the use of any kind of AI in knowledge production—be it a research paper or college essay. 

A second, newer worry is that we are losing the ability to differentiate between human knowledge and AI slop—because of a Luddite refusal to use AI detector tools. This a favourite and expedient peeve of startup founders selling those very tools, like Pangram’s Max Spero. The self-described ‘slop janitor’ first shot to fame by exposing a Guardian sports journalist, then moved on to fiction writers and researchers. Thanks to Pangram, Hachette pulled a horror novel, Commonwealth Prize winners were named and shamed, and so were respected academics and publications. 

The result is a whole new version of cancel culture: 

A pattern emerges: The crowd suspects a problem, then Pangram validates the suspicion, stokes the mob, and sells the solution… In this A.I.-powered asphyxiation of the information ecosystem, Spero has positioned himself on social media as a folk hero hauling in the oxygen tanks. You can tag his company’s bot on Twitter/X, and it will tell you whether a post is A.I. On Spero’s social media to-do list: a “slop hunter of the week leaderboard.”

It’s all deliciously ironic. An AI detector tool is now the ultimate chastity test—outing adulterers who dally with an LLM with a ‘100% AI generated’ tag. The Puritans would be proud. Unfortunately, AI detector tests are often as unreliable as those deployed by Salem witchhunters—at least for now. The percentage of false positives is high. Even Pangram—considered the best of the lot—unfairly damns specific kinds of writers. Example: people who wrote like ChatGPT long before it was born (My name is Lakshmi and I am an em dash addict). Adding insult to incompetence—the tools also fail to identify the worst offenders, such as peer reviews of research papers (see: Nature). 

Pangram’s certificate of purity (100% human written) is likely to become even more dubious—thanks to the efforts of LLM big boys working to make their bots sound human. And as is inevitable, AI detectors tools have also spawned their own jugaadi antidotes. As Atlantic’s Matteo Wong proved in a small experiment, we are caught in a bizarro AI arms race: 

Reddit users rave about a humanizer called Walter Writes AI, which I decided to test out for myself. I had ChatGPT and Claude write brief articles, then pasted them into Walter Writes AI. The program, like other humanizer tools, does some anodyne rewording, swaps one clunky transition clause for another, and introduces grammatical oddities. For instance, ChatGPT’s “The numbers are no longer small enough to ignore” became “The sheer size of these usage figures can no longer be ignored.” When I pasted any output from Walter Writes AI into Pangram, it invariably told me that the twice-baked AI article was human-written.

And ensure the situation is entirely silly, writers are urgently mangling their prose to pass the Pangram purity test: 

“I’ll use aggressively casual language, like, ‘hey yo, for real,’ or drop a bunch of exclamation points,” said [Sarah Suzuki] Harvard, a 32-year-old copywriter in Brooklyn, regarding her posts and essays. “It feels so icky to do this, but it’s what you have to do to sound human.”

Authors are scrubbing out all damning signs of AI crime—even when they have not committed one. No em dashes, no sets of three; yes to run-on sentences and typos—now described as “artisanal craftsmanship.” Then there is the inevitable trend of people unconsciously mimicking AI style because we are exposed to so much of it. Welcome to Absurdistan. 

****

Adding to the addled state of affairs is the ongoing raging debate on social media—which, like any other raging debate on social media, is rife with leaps of logic, confusion of categories and empty posturing. A good example is the recent brouhaha over an open letter penned by the world’s leading economists—Olivier De Schutter, Joseph Stiglitz, Jayati Ghosh, Thomas Piketty, Kate Raworth and Jason Hickel. In it they argue that we live in an age of “manufactured scarcity”—in a system that deliberately creates unprecedented wealth for the few, while stranding tens of millions in poverty. This is the consequence of long-standing and mistaken faith that economic growth over time will shrink inequality (read it here). 

In the past, the letter would have sparked a heated debate over the merits of the argument. This time around, a number of its critics simply copy-pasted the piece into Pangram and declared the letter to be AI-generated. The tactic neatly sums up why the current battle over AI writing is both flawed and futile—let me count the ways.

One: Let’s start with the obvious: Content is not form. Language—whether AI-generated or -assisted—is not equally critical to all forms of knowledge creation. In many fields, the value of a piece of writing is contained in the ideas it expresses, not in the sentences it employs. Pangram may be useful for calling out novelists but it is useless in judging the merits of a scientific paper—which is why we need humans to do that job. Ergo, what should be far more worrying is evidence that more than 50% of researchers use artificial intelligence while peer-reviewing papers. 

Two a): Good writing does not make something good, per se. Pangram puritanism most punishes non-native English speakers who are more likely to rely on ChatGPT for a helping hand—or have learned a by-the-numbers style that can be mistaken for AI. As Alexander Kustov argues, academic English is a common status marker used by gatekeepers to keep out the less fluent. Except now that same fluency has become fodder for AI witchhunts: 

The detector score gives a scientific-looking license to dismiss work without reading it carefully. The people who benefit are usually the incumbent writers and credentialed gatekeepers who can turn a judgment call into a score. The accusation becomes especially convenient against lower-status writers and people who do not write English well but can now use AI to translate, draft, and reach an English-language audience. Too polished looks fake. Too awkward looks low quality. Either way, the gatekeeper wins.

Gatekeepers can do other kinds of damage, as well. Commonwealth Prize judges mistook excruciatingly bad prose (AI generated or not) by writers from the Global South for ‘authenticity’ (Example: “A story is a well. It eats sound until somebody throws a rope”). The scandal rightly shamed the judges far more than the author, Jamir Nazir, exposing the rot within the literary establishment—all of it ‘100% human generated’. The final twist in this tale:

One of the judges, Sharma Taylor, praised Nazir’s story in terms that, to many readers, sounded suspiciously bot-like, remarking on his “precise yet richly evocative” language, which conveys “vivid, lush imagery with remarkable economy.” That Taylor’s statement also conforms to the conventions of creative-writing-seminar plaudits is part of the problem.

Like I said before, garbage in, garbage out. 

Two b): While we’re on the subject of language, let’s also note that Pangram deemed the AI-translated version of the open letter by Pilketty et al published in Le Monde as ‘100% human written’. Yup, AI-tainted in English, but AI-free when translated by a machine into French. What this tells us: we have no clue how AI detectors like Pangram perform in any language other than English. Likely not well.

Three: AI is not just a chatbot. We need better language to differentiate between the many kinds of artificial intelligence, and their applications. A machine that can revolutionize drug testing or auto safety is not the same as the tool that college students or prize-winning novelists use to cheat. In other words, Nvidia’s Omniverse does not equal to OpenAI’s ChatGPT. To put it more bluntly, Pangram aims to solve the least of humanity’s problems—exposing feku authors. Machine-generated writing is also the least of the AI perils facing humanity—since (duh) good English writing is not a requisite skill for 99.9% of jobs in this world. The threat of mass unemployment and societal upheaval lies elsewhere, which is yet another reason to engage with the ideas put forward by economists and not play a silly game of gotcha with Pangram. 

The truth is the purity test is a distraction, perhaps deliberately so. The most important question facing us—as with nuclear fission—is the rightful application of a powerful scientific breakthrough. Tech superstars issue apocalyptic warnings of the perils of AI—including the “risk of extinction,” pandemics and nuclear war. Sam Altman worries he “did something really bad” by releasing ChatGPT, Anthropic won’t release its latest AI model because it is “too dangerous.” But neither Silicon Valley nor Wall Street wants us to think for one moment that we can simply say ‘no’. Decide not to do something just because we can. And that’s the biggest pile of horse shit of them all. 

*****

Lakshmi Chaudhry is Founding Editor at Advisory.