An LLM Is Like A Calculator
Just not in the ways you think.
Thank you for being here. As always, these essays are free and publicly available without a paywall. If you can, please consider supporting my writing by becoming a patron via a paid subscription.
As we continue to grapple with generative AI and its influence, we're understandably looking to the past for guidance. As these conversations have unfolded in education circles, one popular analogy is to compare the impact of large language models like GPT-4 to that of calculators when they began to enter the classroom in the 1970s.
There's been debate arguing both for and against the metaphor, with good reasons for both sides. Today I'll give you my take: LLMs actually are like calculators, just not in the ways you might think.
The Absent-Minded Professor is a reader-supported guide to human flourishing in a technology-saturated world. The best way to support my work is by becoming a paid subscriber and sharing it with others.
Not about how they work or how we respond but how they shape our thinking
Most of the arguments comparing and contrasting LLMs and calculators focus either on how these technologies work or the ways that educators have responded to them.
has a good post on this from a while back on his blog at Inside Higher Ed. He reminds us that whenever we think about technology in the classroom, we need to make sure the learning goals remain the target. Any technology's role must be to support good pedagogy and effective learning instead of simply existing for the sake of novelty.The calculator provides some potentially helpful examples. Introducing calculators too early when learning mathematical concepts can harm learning by creating unhealthy dependencies. If a student doesn't understand how to add or multiply numbers without the aid of a calculator, this reveals an obvious deficiency in mathematical thinking. The same argument applies to doing more complex mathematical thinking like taking derivatives, calculating integrals, or manipulating matrices in linear algebra.
The point here is that math is not at its core about performing computations. Mathematics is a way of thinking about mathematical objects. Unfortunately, this is something that students often don't realize until they reach calculus or sometimes even further. The mechanics of the operations get in the way of the thinking that is really the goal.
At the same time, if one properly understands the power of the calculator as a tool to efficiently execute computations, the calculator becomes a revolutionary tool for extending rather than harming mathematical thinking. When the mathematician is driving the bus, the calculator turns into a tremendous asset.
This shift is made clear when you cross the bridge from theory to application. Once you try to perform operations on very large numbers or large collections of numbers like in matrices, there's no way for a human to accurately perform those computations in any reasonable amount of time. The very same tool that could be a crutch becomes a ladder.
The calculator is a scalpel, the LLM is a Swiss Army Knife
So what does this have to do with LLMs? Transferring this train of thought to how we use an LLM compared to a calculator would mean that we should use the LLM in such a way that it allows us to keep the main thing the main thing. Unfortunately, this is where we hit a snag.
The place where the calculator analogy begins to fray is when you focus too closely on the tool itself instead of seeing the way it shapes the way we think. In short, we lose sight of the fact that the LLM, like the calculator, or any other technology, is a medium. And in the famous words of Marshall McLuhan, "The medium is the message."
When used properly, the calculator has obvious applications. You use a calculator to outsource computations that exceed your ability to easily perform as a human. This might be for any number of reasons like the size of the computation, the accuracy of the result needed, or the speed at which the answer is required. But regardless of the reason, there is nothing fundamentally different about the process that the computer is carrying out compared to the process that a human would carry out if you were to attempt to work out the problem by hand. In this way, the calculator simply extends the power of mathematical thinking. It doesn't replace it.
But once you approach problems with a calculator in hand, you can't help but see problems differently. The presence of the calculator is like a pair of glasses that you can't help but look through. It not only shapes the way you think about the solutions to existing problems, but it shapes (and distorts) the very way that you think about what the problem is. Now even mathematical problems that might best be solved with another method are subject to the calculator. To the one with a hammer, nails abound.
Treat AI like a person at your peril
McLuhan's refrain is getting far too little airtime these days. The LLM is a medium and with it a few taps away in our pockets at all times, it shapes the way we think. Unfortunately, the LLM is an even more sinister case than the calculator because of the way its structure is inherently different from that of a calculator.
One of the first challenges is that LLMs are deceptive. Because they've been trained on human language and we interact with them through text prompts and responses, we're prone to interact with them as we would with people. When we get a text from our mom, we rightfully assume that another person sent that message to us. When we get a response from an LLM it’s hard to avoid the same assumptions.
This is where the problems start, but not where they end. Because LLMs sound so much like us and are built in our image, trained on the corpus of human-written text, one of the best ways to use them is to treat them as if they were a person. It has been shown anecdotally time and time again that giving an LLM a persona in your prompt (e.g., starting your prompt with something like "you are a clever and witty copywriter") can dramatically improve the quality of the responses you get back.
One of the sharpest thinkers today on the topic of generative AI is
who writes the excellent Substack in addition to his scholarly work in this area. He recently published a book, Co-Intelligence: Living and Working With AI1. In the book one of his four foundational principles for working with AI is "Treat AI like a person." He goes on to explain how AI isn't really a person, but it's complicated. At some point in the book Mollick drops the scare quotes around words that anthropomorphize AI like "thinks" or "writes," concluding that even though AI isn't a conscious person, it works better if you treat it like one.I'm enjoying Mollick's book and agree with much of what he is writing. However, at this point, I want to join
and recommend against anthropomorphizing AI. Or at least, suggest that we be very careful. I fear that crossing this line is the equivalent of hopping into the quicksand while hanging onto a branch: it's only a matter of time before we get sucked under.The beauty of the calculator is that it is the equivalent of a sharp pocket knife. A sharp pocket knife is a very useful tool, but it really does only one thing well: cut stuff. In comparison, the LLM is like a Swiss Army Knife. And not just any Swiss Army Knife, but the kind I had when I was a kid which had about 35 different tools on it. Yes, your standard blade and scissors, but a corkscrew, saw blade, awl; to be honest I don't even remember half of the tools on that thing.
Not only is the LLM like a Swiss Army Knife, but we don't even know what all the tools are yet. We're still in the process of finding out where LLMs thrive. But all the while, we are missing the fact that holding the Swiss Army Knife changes how we think.
A fundamental difference between a calculator and an LLM is the difference between deterministic and probabilistic operation. For a given input, a calculator is designed to give you the same output every time. This is sharp pocket knife behavior. LLMs are just the opposite. One of their greatest strengths is that they incorporate an element of randomness so that the same input will give you a different output every time.
This is why LLMs are so deceptive. If you asked a person the same question ten times in a row, you'd probably get ten different, although perhaps only slightly different, answers.
The mistake we make with almost every technology is to fail to see it within its fullness. We miss the forest for the trees. If you want to start seeing the forest again, I recommend picking up a copy of McLuhan's book Understanding Media, where he unpacks this idea in detail. He starts with some foundational ideas and then applies them to various technologies throughout the remaining chapters like the spoken and written word, clothing, the telephone, the radio, and television through this lens.
As you go out this week, consider the way technology shapes your view of the world.
A technology becomes a medium as it employs a particular symbolic code, as it finds its place in a particular social setting, as it insinuates itself into economic and political contexts. A technology, in other words, is merely a machine. A medium is the social and intellectual environment a machine creates.
If McLuhan were alive today, what would he say about LLMs?
Reading Recommendations
Here are a few pieces that I loved this week!
writes an excellent piece urging us to repent of our tendency to treat AI like it’s a person.
An excellent post from
about how to decide on bringing technology into your classroom. Lots of stuff to chew on. writes an excellent piece on the limits of the First Amendment, particularly as it relates to your ability to protest in physical spaces. looks ahead and wonders how all these AI agents are going to impact our economy.
The Book Nook
A friend recommended Elton Trueblood to me and sent me a copy of his book Alternative to Futility. It’s out of print and so a little pricey, but it is available for free on the Internet Archive if you’re interested. I couldn’t put it down and blazed through it in three days. Here are a few quotes that resonated with me.
Science and ethics do not refer to two separate groups of men. Science is, indeed, the work of a specialized class of workers, but ethics can never be. The moral law concerns all of us as human beings, and we are all equally respon-sible. There is not, in ethics, some professional class to whom we can delegate the task of renewal, as we can delegate the task of atomic fission to a specialized class. It is not the moralists who have failed to keep step, but all of us as men and women, and this includes the scientists as well as anybody else.
What we seek is a situation in which the rank and file of our people are filled with a vibrant faith. If they could believe greatly in something, if they could see some purpose to give them a reason for striving, life might become more radiant than we can now imagine. Then, in spite of rubble and lack of housing, men could live joyously and victoriously.
We seek a faith which can dignify the average little life by grounding it in essential bigness, but without divisiveness of class, race or nation. If modern man can achieve a faith which gives his efforts cosmic significance, plus universalistic sympathies, we may have a new world. We could rebuild this planet as rapidly as the Germans rebuilt their little land in the thirties. There have been periods when a vigorous faith has swept like a prairie fire. It is this, and no less than this, which our sagging age requires. But the burning faith might do more harm than good if it were not a faith which involved "liberty and justice for all."
Jesus was deeply concerned for the continuation of his redemptive work after the close of his earthly existence, and his chosen method was the formation of a redemptive society. He did not form an army, establish a headquarters or even write a book. All he did was to collect a few unpromising men, inspire them with the sense of his vocation and theirs and build their lives into an intensive fellowship of affection, worship and work.
a. We cannot have a decent world merely by scientific endeavor. In addition we must have deep moral convictions and a living religion to sustain them.
b. There can be no living religion without a fellowship.
Because mere individual religion is parasitic, there must be a church or something like it, and people who care about the fate of our civilization will join it.
The Professor Is In
We had our final deployment for E80 at Baby Beach in Dana Point, CA on Saturday. The day started out a bit cloudy but the clouds parted around 11 am. Great day celebrating all these students’ hard work!
Leisure Line
Broke out the Solo Stove for the first time last week. Am starting to feel like summer is almost here!
Still Life
The kids had their home science experiment of raising caterpillars over the past few weeks. This last week they emerged from their chrysalises and took flight in the backyard.
The cover of which pictures a motif suggesting that AI is the forbidden fruit.
What is a LLM?
Wonderful essay, Josh. Always glad to see McLuhan brought into the discussion...his provocations are relevant because the transition he describes has been underway since Gutenberg. And thanks for pointing to my review of Co-Intelligence.
I vividly recall getting my first pocket knife, a Barlow, when I was ten. Two blades, limited purpose, but very good for when you needed to cut something. I enjoyed comparing it to my grandfather's swiss army knife and others I encountered, because there was such a variety of tools. I remember loving the chance to flip all the tools out when a friend would let me hold theirs, just to see what it held inside.
I love your analogy because it captures the feeling of using generative AI. You try it out on something and maybe it works, maybe it doesn't. They are both general purpose tools, so the utility and effectiveness vary a great deal with each specific use. The more you use it, the more knowledgeable you are about how to use it well.