An Author Puts ChatGPT to the Ultimate Test

A Familiar Echo of Technological Disruption

I often wonder what went through the minds of 15th-century monks, who painstakingly copied Holy Scriptures by hand, when they first saw a Bible from Johannes Gutenberg's printing press. They believed their role in spreading the gospel was irreplaceable, only to be confronted by a cheaper, faster, and more efficient way.

I can empathize with that feeling. My moment of realization came after reading an academic paper from Columbia University titled "Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers." The title says it all: AI can now write more engagingly than humans.

The Study That Shook the Writing World

The paper's authors conducted an experiment where expert writers from American universities were asked to imitate the style of 50 major authors, including Stephen King, Margaret Atwood, and Maya Angelou. ChatGPT was given the same task. The results were startling—both expert and lay readers preferred the AI's output for its writing quality.

The economics are just as daunting. The paper estimates that a 100,000-word novel generated by ChatGPT costs about $81. For me, writing any of my five published novels was at least a year's work. At that rate, my labor would be valued at around four cents an hour.

Putting ChatGPT to the Personal Test

While the study's authors fine-tuned their AI, I decided to try a simpler approach. I asked ChatGPT, without any tweaking, to "Please write the opening paragraph of a novel written in the style of Keith Raffel."

This is what it produced:

The first time I saw the classified memo, it wasn't the secrets that made me nervous — it was the typos. You can tell a lot about a government from its grammar. A sloppy sentence here, a dangling clause there, and before you know it, someone's invading the wrong country. I'd spent a lifetime parsing both language and power — in Senate hearing rooms, Silicon Valley boardrooms, and, lately, in the quiet of my own conscience. But that morning, with the smell of burnt coffee and ambition in the air, I knew something was off — and that it was going to get worse.

I could guess it was AI-generated due to its overuse of dashes, a well-known tell. But assuming that tic will be fixed, the critical question remains: Would fans of my thrillers keep reading? Though a bit over the top, I suspect the answer is yes.

How much longer can I claim to be the best novelist writing in my own style? ChatGPT even had the nerve to tell me its voice emulated the "Keith Raffel style of weaving the personal with the political, the past with the now." It then had the gall to ask, "Would you like me to continue for a few paragraphs, so it feels like the true first page of a novel — establishing character, place, and the central tension?"

No thanks! The chutzpah!

Can a Machine Capture the Human Pulse

My empathy for those monks deepens. They would labor for months to produce a single Bible, only for a machine to create a more readable version in hours. Technology? Bah, humbug.

Even if you grant that my thrillers are better written than ChatGPT's imitations today, will that hold true next year? Maybe. What about in five years? Maybe not.

I also write this column each week, so I tested ChatGPT's opinion-writing style by asking it to draft a conclusion. Here is its attempt:

So, like those monks squinting by candlelight, I'll keep at it a little while longer — scratching ink onto parchment, so to speak — because the act of creation still matters, even when the machines can do it faster. They may match my syntax and steal my rhythm, but they can't feel the pulse behind the words, at least not yet. And when they finally can? Well, I hope some future algorithm looks back at me the way I look at those monks — with a little pity, a little awe, and a touch of gratitude for lighting the way.

Punctuation aside, it's not bad, is it? It almost makes the machine, or me, seem human.

For Whom the Bell Tolls A Writers Resolve

When it comes to our authorial careers, my fellow writers and I should heed the 400-year-old words of the poet John Donne: "Send not to know for whom the bell tolls. It tolls for thee."

Despite this, I am stubborn and still have something to say. I will continue writing novels and columns because I need to, in a way ChatGPT could never comprehend. I can only hope there will still be readers for them.

---

Keith Raffel is a renaissance man who has served as senior counsel to the Senate Intelligence Committee, founded a successful internet software company, and written five novels. He is currently a resident scholar at Harvard. You can learn more about Keith at keithraffel.com. To read features by other Creators Syndicate writers, visit the Creators website.

Photo credit: Andrea De Santis at Unsplash