The Missing Prompts
Four categories you should consider before turning to AI
Thank you for being here. Please consider supporting my work by sharing my writing with a friend or taking out a paid subscription.
One of the biggest challenges in our current conversations about generative AI is that judging a good or bad use of AI is very context-dependent. It’s not enough if we agree on what the generative AI tool is and what it’s doing. The same tool, doing the same thing, can be helpful in one context and harmful in another.
This week I’ve been noodling about a 2x2 to help me think through some general categories to understand some of the nuance. Here’s where I’ve landed. I’m keen to hear your feedback and whether or not this resonates with you.
A framework for thinking about AI
The fundamental question the 2x2 is trying to address is the question of purpose. Why are you doing what you’re doing? Before we talk at all about whether or not we should use AI, we need to get clear on this.
The two axes try to tease out different dimensions of what’s motivating a particular activity. The horizontal axis describes whether the value is found in the process or the product. The vertical axis describes whether the artifact is instrumental or human.
Let’s dig in and examine each quadrant in detail to get a better understanding of each.
Instrumental × Product: It matters that it gets done
First, let’s consider the quadrant of instrumental work where the product is really all that matters. These are tasks where the details of how it gets done aren’t important. What’s important is that it gets done.
In other words, the value of these tasks is the product they produce, and they are simply an instrument to produce that output. This might be a piece of code that processes some data, provides insight into particular statistical properties of the data, and plots it. It doesn’t matter who wrote the code, or even the specific methods it uses to perform the functionality. What matters is that it gets the job done.
These are application areas where we should consider using AI to speed up our work. If the product is all that matters and AI can enable us to do it better or faster, you should use it.
Instrumental × Process: It matters that you struggled
However, just because a particular task is instrumental does not always mean that the product of that task is all that matters. Sometimes, it’s the process of completing the task that makes it valuable.
Perhaps the most classic example in this quadrant is homework. Homework problems, almost by definition, are not worth doing for what they produce. Most of the time, we already know what the answer is!
The whole point of homework is to work through it, making and correcting mistakes, and learning the right and wrong ways to do something. Homework is still instrumental in that it is essentially a means to an end (to help you build a particular skill), but nonetheless, the process of doing the homework is very valuable in helping you to reach a particular goal and learn.
In this quadrant, AI can be helpful, but only in so much that it retains or enhances the frictionful effort and struggle that is a fundamental part of learning new things.
Human × Products: It matters that you made it
That covers the bottom half of the diagram. What about the top half?
The two quadrants above the horizontal axis describe what I label as human tasks. These are tasks where it really matters that you do it. These tasks are not means to an end, but rather ends in themselves.
A lot of written artifacts fall into this category. A book, a letter, a personal email, this blog post—the product holds a lot of value, but it really matters that a person wrote it. The fullness of human products is found in the way they bear witness to the humanity that produced them.
Human × Process: It matters that you were there
In the same way, there are tasks where the humanity of the process is what matters most. If you have a disagreement with your spouse, the resolution matters, but the way you get there matters even more. The way a team deals with disagreement over a design decision, a family cooking together, a mentor spending time with a mentee, all of these are examples where the process matters more than the product, and the human element is key.
So what? Some questions to ponder when thinking about AI
This framing is helpful in thinking about generative AI in a variety of contexts, but particularly as we think about AI in education. What I find most helpful about frameworks like this is that they can help address ambiguity with concrete examples. A lot of the disagreement about how we are engaging with AI can be traced to different understandings of what quadrant of this diagram we are in.
As we think about the role that AI might play in various parts of our life, perhaps the four different roles in the quadrants above might help provide some guidance. The labels here provide metaphors to describe the tone and nature of the interactions.
In the Instrumental × Product quadrant, AI can be freely used as a tool to complete tasks. The AI tool produces something; we review it.
In the Instrumental × Process quadrant, the AI can be used to help guide our learning, but never allow us to shortcut it.
In the Human × Product quadrant, AI can be used to help us improve what we have created, but never originate it. It can be used as a tool to enlarge our imagination and prompt us to ask critical questions of our work, but never to wholesale generate it without our direct engagement.
In the Human × Process quadrant, AI should have a limited role, if it has one at all. It should be used to help coordinate logistical details when possible, but never to actively engage in any meaningful part of what is going on.
I have a hunch that we significantly underestimate the number of things that we do that fall into one of the two human quadrants. Even something that might seem purely instrumental at first blush, like a homework or term paper, is more fundamentally about human connection than we might realize. We should be hesitant to assume that the things that we do are purely instrumental.
I guess what I’m trying to say is that we should stop to think for a bit before we immediately jump to using AI in every part of our lives. There are some places, like in various aspects of our work, where AI seems to be quite clearly useful and worth leveraging. I’m finding plenty of these places in my own work these days.
But there are other places, like in our homes and in our closest relationships, where the terrain is much more complicated. In these domains, AI should be a part in only very limited times and roles, if at all.
Perhaps before we rush to type (or speak, as the case may be) a prompt into our favorite AI tool, we should pause to answer a few prompts ourselves: What is the purpose of what I am about to do? How will AI support or undermine that purpose? If I do choose to use AI, what role should it play?
Even if the answers to these questions may not be crystal clear, the practice alone of stopping to contemplate them is half the battle.
Got a thought? Leave a comment below.
Reading Recommendations
Thinking more about metaphors and AI as I prepare for a panel that I’ll be on at Baylor next week and a book chapter I’m writing, and stumbled across this piece in Science from Melanie Mitchell in late 2024 this week. It’s a worthwhile read, even (especially?), a year and a few months later.
AI researchers are still grappling for the right metaphors to understand our enigmatic creations. But as we humans make choices on how we deploy and use these systems, how we study them, and how we craft and apply laws and regulations to keep them safe and ethical, we need to be acutely aware of the often unconscious metaphors that shape our evolving understanding of the nature of their intelligence.
I enjoyed this take from JA Westenberg on replacing pitch decks with markdown docs. Worth a read. You can have the best of both worlds and write your decks in markdown and also render them to slide decks with Quarto (tool of choice for moi). PS: I love the design of Westenberg’s website.
If you can’t explain your idea clearly in writing, you probably can’t explain it clearly at all, and if you can’t explain it clearly at all, you probaly don’t understand it well enough to build it.
If you haven’t written your pitch as a plain document, as words on a page making an argument, you’re skipping the hardest and most valuable part of the process. In my experience, the founders who’ve done that homework give better pitches anyway, because they actually know what they’re talking about, and that comes through regardless of whether your slides have drop shadows.
Write the pitch.md first. You might find you don’t need the deck at all, or you might find that when you do build the deck, it’s better, because you finally know what you’re trying to say.
One of my IPAI fellows (h/t Felix) flagged this article on working with Claude Code for us this week.
Read deeply, write a plan, annotate the plan until it’s right, then let Claude execute the whole thing without stopping, checking types along the way.
That’s it. No magic prompts, no elaborate system instructions, no clever hacks. Just a disciplined pipeline that separates thinking from typing. The research prevents Claude from making ignorant changes. The plan prevents it from making wrong changes. The annotation cycle injects my judgement. And the implementation command lets it run without interruption once every decision has been made.
Try my workflow, you’ll wonder how you ever shipped anything with coding agents without an annotated plan document sitting between you and the code.
The Book Nook
Finally cracking open Theo of Golden, a book that first hit my radar in December when one of my Praxis colleagues (h/t Santi) put it in our Praxis Christmas gift exchange. I didn’t take that one home, but it resurfaced again when mom suggested it as the first book for our new family book club. Only a few chapters in, but enjoying it so far.
The Professor Is In
One of the fun things that I’ve been enjoying about Claude Code is how easy it is to take something from idea to reality. Last week, I spent an hour or so and was able to whip out what I think are four pretty cool little teaching demos. You can check them out at teaching.aiprototypes.org and take a peek at the source code in the associated Github repo that is linked at the bottom of the page too.
My favorite is probably this demo on progressive rendering, which illustrates the power of the right set of basis functions and of the Discrete Cosine Transform in particular (the algorithm behind JPEG compression).
Leisure Line
More good pizza in the 16” Ooni this weekend. Finally getting my dough ball sizes dialed in to push it to as big a pie as I can fit in the oven.
Still Life
I was out working in the front yard over the weekend when I heard #1 shriek. I looked over and noticed something slithering along the ground. At first, I thought it was a snake, but closer investigation showed it was a Southern Alligator Lizard, and a pretty big one at that. Grape is there for scale (the little guy didn’t seem interested in having any).
I'm a sucker for 2x2s, and this is an excellent one. Send it to Andy Crouch 😂
Very helpful! Thanks