AI Writing Makes Me Ill

Writing is hard. It is at once a skill that demands pure, heady inspiration and a certain level of pragmatic dedication. It is a rote practice and an idealized striving. Through the process, one transmutes their experience of the world, in all its complexity, into something legible to another. As a craft, then, writing requires attention and care — both to the self and to what one holds within, and toward readers or audiences.

This being said, however, writing is not the product of a talented few. In fact, I could just as easily argue that writing is easy. Children begin to write words in the first grade and continue to learn new skills as they progress through school. Being able to put words on paper is not an innate skill; it is taught and learned — and can be taught and learned well, to the point where everyone passing through a public education system can express themselves in writing. (How this happens is another story....) And as we age, writing remains one of the most accessible forms of self-expression, requiring nothing more than a flimsy ballpoint pen and some newsprint, a dull pencil and some stolen printer paper. (Think about any other art form — photography, film, music, painting — which all require materials and technologies that far outstrip the pen and paper in terms of price and require technical proficiency.) In other words, there are very few barriers to beginning your writing journey and to maintaining a writing practice. Even more generally, we are all, to some degree or another, writers. However, there exists only one real limitation for the practicing writer: time. A writer must have the time to learn, to experiment, to develop, to read; it requires investment by either the individual writer or the society in which they live (hopefully, ideally, both). I even believe that dedication to the craft can be learned: dedication, focus, and patience are, more often than not, skills that require practice. A society either has to dedicate resources to make that a reality for all citizens, or an individual has to overcome that lack of social dedication by teaching themselves.

All of this is to say that if you don’t have or make the time to write, you can’t write — or at least, you probably won’t write well.

But, of course, you can always use ChatGPT.

If you haven’t yet engaged with critiques of the rapid expansion of AI into modern society, I highly suggest that you do. At their most basic, these Large Language Models (LLMs) are essentially a complicated form of plagiarism, using the massive troves of digital writing stored on the Internet to “teach” a machine to recognize patterns in human communication. Insofar as this process constitutes “teaching,” it would more accurately be considered averaging or approximating, with any given prompt being what the machine decides is the logical outcome from the input based on everything that it has processed before.

This constitutes plagiarism because these outcomes have to come from somewhere; and what is fed into the system is the work of writers and journalists and Wikipedia editors and Redditors — all of the written word that humans make online. Put more simply, AI writing doesn’t exist without people, doesn’t exist without writers — so how can it possibly be original?

Now, of course, this “teaching” process requires large data centers, and these buildings take large amounts of electricity and water to operate effectively. As such, systems like ChatGPT have large negative environmental impacts, ranging from basic power and water consumption to pollution, which have further knock-on effects for the communities and economies into which new data centers are thrust.

All of these negative aspects of training and running LLMs could theoretically be managed and perhaps even overlooked if they actually produced anything useful. But, of course, ChatGPT and Claude and all the rest are just barely capable of producing human-like writing. (By comparison, the Internet also runs entirely through data centers and also produces waste heat, water pollution, and requires large amounts of electricity — industrial quantities, in fact. The Internet, by contrast, is actually useful.)

Because of this, companies like OpenAI and Anthropic have struggled to find ways to profit from these models, with OpenAI in particular hemorrhaging money. (It will be worth following the Claude “Legal” extension in the coming months to see if Anthropic has produced something that creates actual economic value.) And OpenAI is the norm: the vast majority of AI start-ups fail, and those that have the capital investment behind them are barely increasing worker productivity, despite all the breathy “AI is coming for your job” fluff pieces.

Now, much more concerning is the investment side of the AI story. It has become quite clear, if not entirely obvious, that AI companies are a large bet made by overconfident tech bros who see LLMs as the wave of the future despite all the evidence to the contrary. It’s a bubble. If we want to take this argument further and look closely at the economic investment patterns of OpenAI, Meta, Anthropic, and NVIDIA, we see what looks a lot more like a scheme to simultaneously inflate the value of the hardware and the stock price of these companies.

At this point, with the absolute wealth of information swirling about the malpractice of these companies, using ChatGPT is capitulating to the very same people who stand to gain the most from the continued inflation of the bubble. Put another way, the program ChatGPT and other programs like it are a Trojan horse: the goal is to manufacture consent (economic, social, political) for “AI” tech, so that when companies like Palantir roll out increasingly dystopian systems of surveillance, warfare, and control, the populace is a little more willing to take it sitting down.

(As an aside, all of this evidence is why I am very suspicious of documents like ai2027. The danger here is not sci-fi aerosol sprays and Terminator-style murder robots; this seems to be pure disinformation. The real danger is the people — Thiel, Musk, Andreessen, and Altman — who hold power in the AI space, who already actively produce AI technologies being used in war zones, and who directly gain from a system intended to control the population. This may seem like a subtle distinction, but it is an important one, for it places the blame on the agents actually responsible: billionaires with their techno-feudalist goals.)

All this is to say that I am very confident in calling generative AI a net negative for society. It regurgitates sub-Wikipedia-level writing so that the user doesn’t have to look outside of one application for information, and essentially off-loads all of the work of thinking onto an external program. (For the record, Wikipedia is good — perhaps one of the most impressive aspects of the modern Internet. Indeed, ChatGPT would not exist without Wikipedia, and certainly could not exist without the vast trove of human data — including yours! — stored on the Internet.) It is not intelligent, it never has been, and if current trends continue, it never will be. The only real demonstrable value of this tech is the continued empowerment of American oligarchs.

Now many people have written and spoken about these issues far more compellingly than I can here. In addition to the sources above, I highly recommend you check out Ed Zitron’s work, several recent Hank Green videos, and basically anything put out by the journalists over at More Perfect Union. They have certainly done the work, and you should support them.

Because of this, “AI Writing Makes Me Ill” is not actually about the negative effects of the AI bubble. Instead, I want to explore AI writing as writing, as a form of communication. I want to think more clearly about the meaning that it produces — or perhaps more specifically, the lack of meaning that it generates. In doing so, I am also going to reference the people who use AI — for it is humans that are always pushing the buttons that make these machines function.

This essay began when I stumbled across an article on Substack. Entitled “Raised by His Softness: On masculinity, care as action, and the gentleness that survived,” the essay reminisces about the relationship between the author and her father. I began skimming. As soon as I did, I noticed a couple of things.

First, I noticed an absolute plethora of references. Twenty-four quotes in total. I have written them out here: Shakespeare, The Merchant of Venice; Camus, The Return to Tipasa; Harper Lee, To Kill a Mockingbird; Wordsworth, “My Heart Leaps Up”; Steinbeck, East of Eden; Seamus Heaney, “Digging”; Cormac McCarthy, The Road; Khalil Gibran, The Prophet; George Eliot, Middlemarch; Jack Kerouac, On the Road; Virginia Woolf, The Waves; Camus, The Myth of Sisyphus; Pessoa, The Book of Disquiet (great pick!); Proust, In Search of Lost Time; Raymond Carver, “Late Fragment”; James Baldwin, The Fire Next Time; Dickens, Our Mutual Friend; Toni Morrison, Beloved; Joan Didion, The Year of Magical Thinking; Hemingway, A Farewell to Arms; Rilke, Letters to a Young Poet; Cormac McCarthy (again), All the Pretty Horses; Khaled Hosseini, The Kite Runner (ew); and finally, a piece that the author calls Shakespeare’s “Song” — but which seems to be a misattributed (and thus plagiarized) line from “When I Am Dead, My Dearest” by Christina Rossetti, which is beautiful. Forgive me if any of the others are also misattributed.

This is a lot of references to include in a Substack essay. While it is not completely implausible that a “20-something forensic psychologist” would have read this range of literature, being able to curate a collection of quotes this large and integrate them into a lengthy essay every single day is not. For indeed, theinkspilled posts essays of this complexity routinely, every day. A reasonable conclusion: ChatGPT collected a set of quotes about fathers and simply dumped them into the essay theinkspilled prompted it to create.

This conclusion seems pretty reasonable, especially considering that these quotes regularly lack proper contextualization. When a writer includes quotes in their work, they will probably first explain their context within the work they are citing, and then they will probably say something specific about them. Maybe the passage bolsters the argument being made; maybe its inclusion informs the overall perspective of the piece. Even in a more creative, personal essay, the author should acknowledge the quotes they include, even if only to offer a brief explanation of their origin.

Compare this description to a passage from theinkspilled’s essay:

There is a line from Harper Lee that feels almost embarrassingly exact, “Atticus was right. One time he said you never really know a man until you stand in his shoes and walk around in them,” from To Kill a Mockingbird, and I think now that what I knew of my father then was not heroism but endurance. And from Wordsworth, quieter and truer, “The Child is father of the Man” from My Heart Leaps Up, because in saving me, he was also repairing the lonely child he once was.

Some of the analysis here feels plausible, but I would certainly not describe it as clear. (And why would it be embarrassing for this quote to be “exact”?) Further, the references do not really follow from each other. This feels very much like ChatGPT padding an essay. Because ChatGPT does not understand why authors reference things — does not understand the relationship between different sources of information beyond their most basic definitional similarities — it mostly errs toward curation and basic clichés. It does not actually analyze the quotes; it merely approximates the feeling of analysis.

The second thing I noticed was a style that feels appropriately warm but, upon closer reading, is actually quite superficial. Here’s an example.

Growing up, the house often sounded like the weather. My mother’s pain, which I would only later learn to name as illness and exhaustion and despair, moved through the rooms like a storm. Raised voices, doors, the kind of tension that makes a child hold her breath. And then my father would appear at the edge of my doorway, quiet, almost apologetic for the noise of the world. He would sit on the bed, kiss my head, and tell me everything was going to be okay. Sometimes he stood between me and her anger, not confrontational, simply present, like a wall that would not move. As a child, I thought of him as a knight, something medieval and absolute, the one adult who seemed built for protection.

This is not necessarily bad writing per se, but it lacks the specificity that you might expect from such a personal piece. The ideas here work at an ephemeral level, often capturing the feeling of being raised by a caring father, but as a whole the essay never really says anything about those feelings. We get a sense of the author’s father, but we never really learn who he actually is. He’s a builder; he has lived a hard life; he completed a thesis; his hands are rough from work; he is a carer; he struggles with sleep; his wife is ill; and none of this is made specific. The whole piece feels a bit like light filtered through clouds, like the shadow of an embrace cast on a beige wall. Again, it feels like approximation.

In sum, I felt that the piece lacked perspective. There is no obvious thesis statement or narrative purpose beyond the general idea of being raised by a caring father. As such, there is no real resolution:

If my mother gave me language, he [the author’s father] gave me ground. And every time I look at him, I think the same simple, unphilosophical thing. I was lucky. I was always, unmistakably, my father’s daughter.

These final lines feel correct — like they are the correct words with which to end this type of essay — but they are not actually symmetrical with the rest of the piece. If this final idea was the point, the center of the essay, it was never properly established. As such, “Raised by His Softness” is, at best, a nice idea entirely under-explored. It feels like the idea of an essay rather than an essay. “Raised by His Softness” feels like the product of generative AI.

When I get suspicious that a bit of writing has been AI-generated, I prompt ChatGPT to write something similar. I put myself in the shoes of the author. In this case, I wrote the following prompt:

“Write a personal essay about the care of a father for their daughter, using personal experiences and many quotes from popular literary fiction to explore the idea.”

Now I will be clear here: there is a rather obvious bias in this experiment; I am certainly working backward from my conclusion that theinkspilled used AI to write the essay. However, if we use this method, we can at least draw some comparisons to get a sense of the process. More importantly, we can also analyze the “personal” experiences that ChatGPT generates. Through this analysis, we might learn to better recognize AI writing — as attentive, human readers. And most importantly of all, I hope to explain exactly why AI writing is bad — and why it makes me ill.

So, when I ran my prompt, ChatGPT immediately pulled the same Shakespeare quote that theinkspilled used: “It is a wise father that knows his own child.” I had to run it a couple of times to make it add in so many references, but when I did, many of the other quotes appeared. (When I asked it more specifically to simply aggregate quotes about fathers, it generated a very similar collection that existed in “Raised by His Softness.”) And of course, as I skimmed what it “wrote,” I found the same reliance on bland imagery that we saw in “Raised by His Softness” — porch lights, car rides, caring gestures, the calming silence of a male role model.

But most obvious was its tone. ChatGPT and similar models are often quite sycophantic. This agreeability is actually deadly — that ChatGPT coached a 16-year-old through their suicide — and has led to a whole swath of cases of “AI psychosis.” There are extreme effects of LLMs on the psyche. Yet this tone is not only interesting for what it does to the vulnerable; here, I want to interrogate this agreeable nature as it exists in the writing itself — and particularly show just how uncanny AI writing can be.

From the essay I prompted it to write:

When I think of our [the AI’s and the AI’s father’s] relationship, I think of long car rides. The hum of the highway. His hands at ten and two. The way conversation unfurled more easily when neither of us had to look directly at the other. There is a particular intimacy in shared forward motion. He would ask questions that seemed casual but were anything but: What are you reading? What are you thinking about lately? What do you want?

Certainly, there is a banal emptiness to this passage. It is a relationship mimed, without any of the detail we would expect from a person with an actual, presumably complex relationship with their father. But as it continues, the passage becomes increasingly eerie. The “intimacy of shared forward motion” is an interesting touch, but the comment about the ease of conversing without looking at each other feels impersonal; it hints at the obvious question: why can’t they look at each other normally? A human writer might interrogate this question or the feeling, might lament the discomfort they feel when under their father’s gaze, or might discuss the emotional effect of watching the road scroll by while chatting aimlessly. And of course, the end of the paragraph — “What do you want?” — feels actively accusatory. We have no context for this question; it is as if the question falls to the reader to answer. It is an incredibly odd construction, even as it feels correct.

And from “Raised by His Softness”:

He worked until work erased him. Evenings ended the same way. Dinner, the television humming, and then sleep took him mid-sentence on the couch. Not rest, but collapse. I would shake his shoulder gently and tell him to go to bed. Sometimes he didn’t make it. Sometimes, insomnia pulled him back awake, and he walked through the house in the dark.

The same emptiness is present here. The author does not comment on what exhausted their father. She offers no thoughts about his class condition, on the struggles of keeping together a family under economic strain. Neither is he a father with any specific passion for a work that drained him. He is not a nurse or a teacher. He works until he literally wanders the house out of pure restlessness — like an exhausted ghost. There is, more simply, nothing to learn from the author’s father’s unending work. “He worked until work erased him,” writes the author, and then fails to describe the erasure. I am left with only two conclusions: either I have too little context to really understand this man’s experience, and thus the essay is poorly written; or this erasure is intended as vaguely positive, an idea which leaves me with a sinking feeling. We must keep working, the essay implies; this makes for a good father. In the end, our author has apparently been “raised by his softness,” but we get a paragraph that implies a deep disquiet.

Then the paragraph ends with several quotes disconnected from their context. From East of Eden and The Road, they are both superficially connected to fatherly labors — with the quote from The Road in particular feeling out of place considering the absolute desolation of that novel’s setting. So as the paragraph rounds the corner into its off-topic conclusion, it becomes clear that the personal touch at the beginning of the paragraph is mimed to pair with the quotes from later important texts.

Or, in all actuality, ChatGPT began a paragraph about work being exhausting and then found some superficially similar quotes to finish the paragraph. Considering the linear direction of LLMs, this makes far more sense: to complete its programmed task.

LLMs are an inherently sinister force on our planet. Not only do these programs already have a body count (and maybe more than one...). Not only do the vulnerable succumb to ChatGPT’s programmed sycophancy. Not only do data centers pollute our communities and enrich the wealthy. Not only does it steal from human authors.

No, LLMs are evil all the way down: the banality of their generated writing is itself sinister. When people like myself say that ChatGPT isn’t human — that it doesn’t have a soul, that it doesn’t “know” what it creates — they don’t just mean this as a metaphor or to differentiate its writing from human writing. It is a far clearer statement: any reading actually attuned to the intention and emotion of an AI “essay” reveals that AI fundamentally misuses humanity. It fails to interrogate its claims, it lacks specificity, and it seems entirely unable to stick to its main point. Even at its best, as it curates or summarizes quicker than any human could, ChatGPT can only do so by using basic associations, by approximating what two ideas go together based on what most essays put together. In so doing, it perverts the actual relationships between ideas. AI writing is all feeling, no meaning. It is the idea of an essay, not an essay itself.

More importantly, the proliferation of unlabeled AI writing on Substack and the Internet at large creates two opposing sinister forces: a general suspicion that earnest readers will now harbor toward all writing they read online; and the reduction of the purpose of writing to superficial aggregations of our social and cultural world. For of course, as AI continues to be used, and more AI writing gets shuffled into the mix with human writing, the more people are going to become accustomed to generated text. They will see it as normal, as legible, as meaningful. As I have shown above, it does not actually produce meaning. It produces the appearance or feeling of meaning. And thus, the suspicions of serious writers and readers will multiply.

This reality is not merely an unfortunate side effect of an otherwise useful technology. It is the only logical outcome of the sinister averaging and aggregating forces inherent to LLM technology.

And all of this is sickening. AI writing makes me ill. It makes me ill when it mimics human emotions. It makes me ill when it tries to construct a relationship between two of my favorite novels. It makes me ill when it tries to formulate something that approximates experience. When it pretends to understand the value of a caring father, when it pretends to have had a conversation in a moving car. It produces marionette words, but no hands pull at the strings, so they fall to the floor, a loose, limp, inanimate pile.

Writing is hard. It takes time and dedication and practice and everything in between. It requires a strong social safety net, a society willing to give every child the right to read and write and search for meaning beyond the banality of their work and daily survival. Writing is a striving for more. And it is that striving that makes it difficult — so difficult.

So when one prompts an AI to strive for them, to write about their experience for them, they abandon themselves. There is no way around this; they abandon themselves. It is this abandonment that makes my stomach hurt, that makes me deeply depressed about the state of our collective intellectual life.

So. I will remind anyone who reads this essay that your thoughts, your experiences, your life... it is all worth writing about, in all its messy specificity. Do not offshore your humanity to a machine; do not allow ChatGPT to average your experiences, to pervert your interests, to sand you down.

Instead, do the hard thing, the rewarding thing, and write.