Process Over Product
How systems-level thinking can help us wisely engage with AI in our writing
“To succeed once, focus on the outcome; to keep succeeding, focus on the process that makes the outcome.”
Every writer needs to grapple with the blank page. There’s no stage of the writing process quite so full of possibility and excitement. And yet, producing the first draft is often also the hardest part of writing.
Generative AI offers a tempting solution: outsource the hard work to the machine. Tell it what you are trying to say and let it get you started. It’s a tempting offer. But we’d best think carefully about why we want to solve the problem of the blank page. I’m afraid that in many cases, we are conflating pain with strain.
Strain is good. Pain is not.
If we stop writing first drafts we might be eliminating some pain but we’re going to wipe out a lot of strain along with it. This is not a good thing.
Today I want to dig into why we should embrace the strain of first drafts. Listen, I know writing ain’t easy. But not many things worth doing in life are. Writing is hard because thinking is hard.
While writing might be hard, deconstructing the process as an engineer has helped me to understand my own process of writing, learn how I can build systems to support it, and highlight the questions I should ask when I think about adding technologies to the mix.
The process of writing in block diagram form
If you study creative people for a while, one of the things you’ll notice is that they almost always embody a prototyping mindset, even if they don’t use those words to describe it. At its core, the prototyping mindset is about applying a design process: empathize, define the problem, ideate, prototype, and test. Then do it again and again, learning from your failures each time.
Systems-level thinking can help us understand how the iterative feedback loop present in design thinking works. Systems-level thinking is about breaking a process down into its individual components and modeling the relationships between them. This makes it easier to understand the behavior of the overall process by understanding its parts. It is a powerful tool in the context of engineering design and analysis but it’s also very useful for understanding ideas outside of engineering.
The block diagram is one of the most powerful tools of systems-level analysis. You’ve probably seen block diagrams before, but here’s a little bit of the terminology to orient you.
The boxes represent operations. They take in an input and produce an output.
The lines that connect the boxes represent signals.
The diagram moves from left to right from input to output.
At its core, the prototyping mindset is a feedback loop. This is the secret to its power and also its flexibility. The diagram below is a general representation of a feedback loop in the form of a control system and it is a helpful template to apply to many different systems. Let’s see how it might help us model the process of writing.
Here’s how the system works. You start with a desired output and compare it to the feedback signal which contains information about the actual output. Here the desired output is your vision for what you want to say and the actual output is the most recent draft. The circle with the Σ in it computes this error which is simply the difference between these two quantities. This might describe the ideas you have to improve your writing. For example, maybe you want to add more data, improve the flow of the piece, or trim the piece to make it concise.
The controller takes in the error and determines what to do with it. Which of the pieces of feedback do you want to address? How do you want to change the writing? Do you need to add or cut text? Where could you tighten your logic or strengthen your argument?
Once the controller decides the action to take, then this action is applied to the process. In our writing example, this is where you sit down and start writing, moving from abstract ideas into concrete concepts expressed in the text. The output is the text of your essay.
The last part of the diagram is the measurement block in the bottom path. This is the most critical element. Without this block, our system is open loop instead of closed loop. In an engineering context, open loop systems are bad news. It’s just really hard to design a system that does what you want without feedback. In the context of writing, this measurement system might represent any of the many ways that we might get feedback on our work, for example, re-reading our own writing, having a peer or colleague review it and provide feedback, or using a generative AI tool like ChatGPT to give us suggestions for improvement.
A writing block diagram
To put it all together, here’s a fresh copy of the diagram with each part of the diagram replaced with its writing-specific signal or task. Seeing the work of writing in this process is helpful for a few reasons. It helps us to understand…
The specific elements of how we write. Writing is much more than the act of putting words together on a page. Writing is thinking: figuring out what to say, why to say it, how to say it, and then assessing if the text we generated actually says the thing we want to say.
The impact that a particular technology might have on our writing. If I change from writing by hand to writing on a word processor, this doesn’t have any direct impact on the measurement or controller blocks, but it does change the process block. In a similar vein, getting feedback on our work in different ways doesn’t necessarily change the way we decide to act on it but it does change the information available to that process.
The importance of iteration. Writing, like any craft, is not a one-shot process. Becoming great requires dedication, repetition, and lots of failure along the way. Your first performance is not your best. Your first essay is not your most persuasive or creative. The best stuff is the evidence of many hours of intentional hard work behind the scenes.
The importance of knowing what you want the output to look like. Even if all the parts of your writing process are working well, they’re only as good as the overall input to the system which represents the vision you have for the work. This vision has multiple components including the facts, rhetoric, style, and arguments present in the written product.
Learning is an iterative process
It’s worth spending a bit more time unpacking why iteration is so critical. In short, it’s because learning is a cyclical process of active engagement and thoughtful reflection. The learning process is what helps us grow and improve in our craft.
I was reminded of this recently while listening to a recent talk from
. In it, he shared three stories from his journey and how they’ve shaped his approach to his work as a professor. His core value as an educator hits the nail on the head:The only way that human beings learn anything important is by actively engaging in the learning process and actively reflecting on what you’re doing.
If there was ever a thing to be laser-focused on as an educator or a student, this is it. If we want to learn, we need to engage actively in the learning process and reflect on the work we are doing.
This idea, combined with a systems-level view of the writing process explains why we need to be careful about how we embrace tools like AI in our learning processes. If we understand what the tool is doing for us, we may be able to use it wisely. But if we treat the writing process as a magical black box, then we’re not setting ourselves up for success.
Don’t hand over the keys to the co-pilot just yet
With this background in mind, let’s return to the question of writing and first drafts.
laid the situation out pretty starkly in a recent tweet.
I don’t doubt that Ethan is right here. I’m sure that many will end up using an LLM to avoid the stress of the blank page. There are ways to use AI to help you create a first draft, but just handing off the whole thing is a mistake. The block diagram from earlier can help us see why.
Using many of the first-generation AI tools like ChatGPT to generate a first draft requires us to actively resist their default configuration. When the generative AI feature in Google Docs says “help me write” with a shiny button and a magic wand or ChatGPT presents you with a text field to ask it a question these tools are encouraging us to offload our mental burden.
The temptation here is to give the machine full control and check out. It’s almost exactly the same failure mode as a teacher or tutor solving the problem for you instead of walking you through how to solve it. If we’re not actively engaged in the learning process, then we’re not going to learn anything.
Through the framework of our block diagram, we’re letting the machine do both the feedback filtering and text generation steps. We describe our desired output (the prompt) and then the LLM takes it from there. Sure, we can still reflect on the output to suggest changes in a follow-up prompt, but we are still treating the process from idea to text as a black box without any real control over what is going on within it.
What we do know is that there is no thought going on in the process, just the generation of highly probable combinations of words according to modeled syntax rules. We have no idea what sources are being considered, how they are being weighed against each other, how the sources have been put in conversation with each other, what particular biases might be present, or any other detailed aspects of the writing process. All of these things and more are critical parts of thinking and writing well.
Some may see this black box model as good news, but I don’t count myself as one of them (and to be clear, I don’t think Ethan does either). It’s true that hammering out a first draft is one of the hardest parts of writing. I feel this every week when I head over to Substack, duplicate the template post from my drafts, and sit down to figure out what I want to say.
But writing is not just about the text that you write at the end of it. Writing is about the process that leads you to that eventual end product. It’s about iteration and failure: the cycle of drafting, critiquing, editing, and revising. It’s the same underlying truth about any worthwhile pursuit: it’s about the journey and not just the destination.
The question we’ve got to be asking about LLMs or any other technology we use is how it supports not just the surface-level artifact we’re trying to create (e.g., an essay), but whether it helps or hinders the process that helps us reach that goal.
Three heuristics for writing with an AI co-pilot
Outsourcing the whole writing process to AI is a mistake. However, there are thoughtful ways to include it in your process. Here are a few ideas to consider:
1. Target your use of generative AI
Don’t use generative AI for more than one specific part of the writing process.
Do experiment with AI as a targeted aid for individual portions of the process. Use it to help you brainstorm ideas to consider new angles, ask for editing suggestions, or try asking it to write an idea in a particular style.
2. Put not your trust in LLMs
Don’t take your hands off the stick. AI might be your co-pilot, but it’s not trustworthy. And even if it gets more trustworthy it’s not you. And so that means it can’t say what you think. You should imagine that you’re in one of those cars with two sets of pedals. Always be ready to pump the brakes.
Do remain in active control of the process of writing. Continue to think critically about what you want to say and how you want to say it, even as you explore how AI tools might help you to write more effectively.
3. Remember what’s under the hood
Don’t forget what’s going on inside an LLM. While the capabilities of these programs are impressive, remember that they are simply very large computer programs that are computing probabilities to generate text.1
Do use the LLM to your advantage. Ask it to generate ideas that counter the text you’ve written or suggest how someone who disagrees with you might respond. Ask it to generate large quantities of ideas to break you out of a rut and help you to consider different angles.
The bottom line: Eliminate pain. Embrace strain
Here’s the long and short of it: learning is hard. Thinking clearly is hard. Writing well is hard. So when a new technology comes along that promises to help alleviate some of that burden, it’s only natural that we’re hopeful.
But in our desire to find some help, it’s easy to lose track of the goal. Do we just want to do the same things we’ve always done faster? What’s the point of all of our striving to begin with?
These questions are fundamental to our use of any technology, but they are particularly relevant when we are considering the process of writing. Writing is so closely coupled with what it means for us to think deeply. If we aren’t thinking clearly about what we’re doing and why, then we’re already in trouble.
Learning is not about pain, but it is about strain. If we’re not feeling some tension and strain then we’re not learning and we’re not growing. As we think about how to approach writing aids, whether that be LLMs or word processors, we have to keep this important distinction front and center. If we’re eliminating the strain we feel when we’re learning and growing, we’re not doing ourselves any favors.
Thoughtful AI Engagement in the Writing Process
If you’re looking for a tool to explore generative AI in your writing process Lex is worth a look. This word processor from Nathan Baschezand his team is a great example of how AI might be an integrated part of the writing process in a targeted way. At its core Lex provides you with various ways to interact with generative AI in your writing. But it does a good job of helping you avoid the ambiguity of “help me write.” Instead, you as the author are encouraged to use the tool for specific tasks like improving clarity, making suggestions about flow, and analyzing weak points in arguments. You can read more about it or try it out here.
Further reading
If you’re trying to think about ways to experiment with AI in your writing process, here are some places where you might find inspiration.
- had a great post on his Substack last week where he shared a survey of some tools that are being developed to thoughtfully interface generative AI with the writing process instead of relying on the defacto chatbot interface.
- wrote about his experience creating his GPT doppelganger. While I wonder how GPTs might be used to create a tool that is focused on a specific task in the writing process (e.g., design it to focus on brainstorming lists of potential ideas without fleshing them out in full paragraphs of text), John’s experience offers insight into the ways that machines might sound like us but ultimately are not us.
Anna Mills continues to be a great source for thinking about how you might engage with generative AI in your writing process. She recently shared a slide deck from a recent talk which has some great background information and some microlessons on how you might incorporate AI into your classroom.
The systems-level view of writing also demonstrates why it’s so important that we read widely and get good writing instruction. To know what we want our writing to look like we’ve got to have a clear conception of what good writing is. The best way to do that is to read widely and thoughtfully with an eye toward how the writers we admire execute their own writing craft.
Got a comment? Would love to hear it!
The Book Nook
The answer, it turns out, is yes. This is well worth your time and an especially good gift for students who are strapped for time but are hungry for advice on living well.
Here are a few of my favorite quotes from the book.
A superpower worth cultivating is learning from people you don’t like. It is called “humility.” This is the courage to let dumb, stupid, hateful, crazy, mean people teach you something because despite their character flaws they each know something you don’t.
To be interesting just tell your own story with uncommon honesty.
If we all threw our troubles into a big pile and we saw everyone else’s problems we would immediately grab ours back.
The best way to advise young people is to find out what they really want to do and then advise them to do it.
As long as an idea stays in your head it is perfect. But perfect things are never real. Immediately put an idea down into words or in a sketch, or as a cardboard prototype. Now your idea is much closer to reality because it is imperfect.
Don’t define yourself by your opinions because then you can’t change your mind. Define yourself by your values.
Very few regrets in life are about what you did. Almost all are about what you didn’t do.
The Professor Is In
Last week my students in E155 updated me on the progress on their final projects. It’s exciting to see them come to life and I can’t wait to share the final products with you in a few weeks!
Leisure Line
This week is Thanksgiving in the US and I’m looking forward to taking some intentional time to reflect on all that I’m thankful for.
Last year I started what I think is going to be a new annual tradition of making
’s Thanksgiving gratitude zine. Here’s a photo of the one I made last year. If you’re looking for something to do between dinner and dessert, print a few of these out to make with your family and friends.You can download the files from Austin’s Substack at the link below. If you end up making one, I’d love it if you’d share a picture of it with me!
Still Life
As I’m driving around town I’m always on the lookout for new-to-us toys for the kiddos. This last week we scored a Thomas the Train Engine set that became an instant hit.
@Josh, it is an amazing post again! Your three heuristics are very useful, I cannot agree more on keeping the thinking part for your own - on this aspect, I like Shane Parrish's "writing is thinking" mentality, which I also reflected on in one of my newsletters: https://path2phd.substack.com/p/p2p-no-34-on-ai-based-tools-for-writing