Mens Sine Manus

Why AGI can't deliver on its promises

Thank you for being here. As always, these essays are free and publicly available without a paywall. If my writing has been valuable to you, please consider sharing it with a friend or supporting me as a patron with a paid subscription. Your support helps keep my coffee cup full.

---

“The Philistines Place the Ark of the Covenant in the Temple of Dagon” Battista Franco, Italian ca. 1540 (image source). Depicting the story from 1 Samuel 5.

The idols of the nations are silver and gold,
made by human hands.
They have mouths, but cannot speak,
eyes, but cannot see.
They have ears, but cannot hear,
nor is there breath in their mouths.
Those who make them will be like them,
and so will all who trust in them.

Psalm 135:15-18

Artificial General Intelligence, or AGI, is in the news again this week.

Ezra Klein

says the government knows it’s coming. The Manus AI team says its new agent is a “glimpse of AGI,” showing how it can help to screen and rank job candidates without any human interaction.

My favorite is the line where the Manus co-founder and chief scientist doing the video voiceover talks about how “Manus carefully reads articles” to compile a document. No such thing is actually going on. As

Zvi Mowshowitz

wrote yesterday, “It’s mostly a wrapper on Claude that uses a jailbreak prompt, 29 tools and browser_use, with what everyone does agree is a very good UI.” Huh.

Part of the problem is that AGI is an ill-defined term. In an essay on OpenAI’s website in February 2023, Sam Altman defined AGI as “AI systems that are generally smarter than humans.” Doesn’t sound very easy to quantity to me, but fine. But let’s take a brief pause from arguing over what it is and talk about what it is for. In the next paragraph of his piece, Altman writes:

If AGI is successfully created, this technology could help us elevate humanity by increasing abundance, turbocharging the global economy, and aiding in the discovery of new scientific knowledge that changes the limits of possibility.

AGI has the potential to give everyone incredible new capabilities; we can imagine a world where all of us have access to help with almost any cognitive task, providing a great force multiplier for human ingenuity and creativity.

Now we’re getting somewhere. This is about what AGI does for us. What he takes for granted is that increasing abundance and turbocharging the global economy by definition will elevate humanity. Perhaps, depending on what you think humanity is for.

What if we are glimpsing AGI?

For the sake of argument, let’s assume that Ezra Klein and his insider sources are right. Assume AGI, whatever it is, is around the corner—what then? What are the questions we should be asking as we prepare? At the end of his interview with Ben Buchanan about AGI, Ezra gets a bit irascible—enough so that he lets the rare-for-Ezra expletive fly. He pushes Buchanan:

You start the conversation about how the most transformative technology — perhaps in human history — is landing in a two- to three-year time frame. And you say: Wow, that seems like a really big deal. What should we do?

That’s when things get a little hazy. Maybe we just don’t know. But what I’ve heard you kind of say a bunch of times is: Look, we have done very little to hold this technology back. Everything is voluntary. The only thing we asked was a sharing of safety data.

Now in come the accelerationists. Marc Andreessen has criticized you guys extremely straightforwardly.

Is this policy debate about anything? Is it just the sentiment of the rhetoric? If it’s so [expletive] big, but nobody can quite explain what it is we need to do or talk about — except for maybe export chip controls — are we just not thinking creatively enough? Is it just not time?

Ironically enough, Ezra’s own interview with

L. M. Sacasas

from a few years ago has at least the beginning of the answer. At the end of that interview, Ezra presses on Michael with a similar question: “Can I be held responsible for the actions which this technology empowers? Would I feel better if I couldn’t? Could you talk about that, and a bit about how you think of the ethic of responsibility in technology?”

There are multiple ways in which we’ve structured modern life in such a way that it’s easy to say this isn’t my fault, that the machine made me do it, the process made me do it, the system made me do it, where I think this is ultimately not really conducive to a healthy society. I don’t know what the answer is, once we’ve already kind of put these structures in place. But I think it’s important to be able to locate moral responsibility as much as possible, and especially if I don’t want it to be attached to me.

The first step is the intellectual humility to say, “I don’t know what the answer is.” The second is to locate the moral responsibility. A good place to start on that journey is Sacasas’s own post from almost four years ago, “The Questions Concerning Technology.”

What is our moral responsibility?

Before we can even begin to unpack what our moral responsibility for AI or AGI is, we need to wrestle with who we are and why we’re here. If there is no purpose to being human, then it’s a relatively short conversation. Eat, drink, and be merry, as they say. Perhaps we won’t die tomorrow, but we’ll soon enough hand over the keys to the AI that will finish the job.

But if you believe that there is meaning to life and purpose to what it means to be human, we should think carefully about our next steps.

There are some potential parallels to where we are with AI. The development of the atomic bomb is one. The development of techniques that enable human cloning is another. In each of these instances, the collective intelligence of humanity has the ingenuity to discover and build technology with great potential for both good and evil. On the one hand, the path to energy abundance and the eradication of human disease. On the other hand, the fork in the road to mutually ensured destruction and bioweapons.

In both cases, we had the foresight and situational awareness to look down the road at the various futures ahead of us and choose a wise path. We may be nearing that junction with AI. But are we too busy trying to run fast and staring at our feet as we rush headlong over the cliff’s edge?

What would it mean to pick up our eyes and search for the horizon? Indeed, part of the challenge is the fogginess of the way ahead. The technological explosion of generative AI over the last few years has done little to make the path ahead any easier to predict. And yet, there is still a way to move forward wisely.

The Power of Questions

Instead of the pat answers and overconfident responses that we’ve grown accustomed to seeing from LLMs, we’ll need true humility. We’ll need to recognize what we know and what we don’t and develop the wisdom to know how best to move forward in the presence of many unknowns.

This is where Sacasas illuminates a path forward. What we’ll need more than any particular set of answers is a robust list of questions. Questions that can guide us with wisdom. Questions like:

• What does it mean to be human?

• Do humans possess intrinsic moral worth irrespective of any economic value?

• What is the purpose of life?

• What is the purpose of work?

• What does it mean for us to be embodied creatures?

• How ought we treat our fellow humans?

• What does it mean to flourish?

• What are our guiding values?

The road ahead may be full of unknowns, but I can tell you one thing. AGI won’t deliver on its promise. No matter how much it increases our efficiency or boosts our productivity, it won’t satisfy the deepest longings of our souls. What it means to be human is about more than just material abundance. Life is about more than creating more with less or freeing ourselves from needing to work to sustain ourselves.

What Sabbath and the Imago Dei Can Teach Us About AI

Many concepts from the Judeo-Christian worldview are beautiful to me, but two of the most beautiful are the ideas of Sabbath and the Imago Dei. Both of these provide powerful answers to some of our fundamental questions about AI.

The Imago Dei means that as humans, we are made in the image of God. In some mysterious way, we are all stamped with characteristics that make us intrinsically valuable, possessing value to the creator of the universe simply because we are his creation. It means that we are loved by him and designed to rest in him. That the value of our life is simply because we are, not because we do.

There is perhaps no better reminder of that truth than the concept of Sabbath. In the story of creation, as poetically recounted in Genesis, God creates the world and then rests. What he is showing us is the goodness of rest. Although in his infinitude, he didn’t need to rest, he knew we would. And so, he reminded us of our dependence on him, a dependence that like any dependence is a limitation, but which in an upside-down way, is the only path to true freedom.

In the Jewish tradition, the Sabbath was the seventh day of the week, bringing a close to the busyness of six days of work and toil. As Christians, we celebrate Sabbath not as the seventh, but as the first and the eighth day of the week. We celebrate it as the first day of the week because Jesus was raised to life on a Sunday. On the eighth day, we are reminded of the blessing of rest.

The practice of Sabbath forces us to confront our own limits. Ironically, this is a gift that the power of technology, and even of AGI, can give us also. Technology reminds us each and every day that we are frail and fallen creatures. That we have weaknesses and are not infinitely powerful, wise, or good. That we need rest.

The great promise and deception of superpowered AI systems is one and the same: rest. AI in its various forms offers to free us from toil. To give us a life of ease and abundance. But it will not deliver on this promise. If we choose to give ourselves over to it, we will find ourselves enslaved by it.

The deepest longing of our hearts is not for material abundance but for rest. There is only one place where can truly find it.

If you’ve gotten this far, you might wonder what the title of this post is all about. What exactly does mens sine manus mean? It’s a statement of defiant disagreement.

In their video showing off their new tool, the folks over at Manus AI talk about how the name comes from the Latin motto Mens et Manus, “Mind and Hand.” This name, they say, is meant to convey the power of the Manus AI agent “to be the hand that brings your mind’s vision into reality.”

The irony, of course, is that this AI agent is like all the others. It has no hands. Or mind, for that matter.

If we’re paying attention to the real rhetoric behind the quest for AGI you’ll notice that the situation is exactly the opposite. The reality is not that AGI will be the hands for our minds but that we will be the hands for it.

Let’s be clear-eyed before we hand over the keys.

Come to me, all you who are weary and burdened, and I will give you rest. Take my yoke upon you and learn from me, for I am gentle and humble in heart, and you will find rest for your souls.

Matthew 11:28-29

Got a thought? Leave a comment below or shoot me an email.

Reading Recommendations

I had the wonderful opportunity to meet Punya Mishra last week over lunch while he was on campus to give a talk in Claremont. In doing so, I stumbled across a kindred blogging spirit (although not on Substack—yet!).

Punya’s post from last month about AI’s impact on expertise is well worth your time. This figure about AI’s impact modulated by different levels of knowledge is also very thought provoking. I’m hoping to find time to record a conversation with Punya sometime in the next few weeks, so stay tuned for information about that if we can find time to make it happen!

Last week

Zvi Mowshowitz

wrote a piece about school that’s well worth your time. It was a particularly interesting read for me through the lens of my own experience as a homeschooler and now making decisions about how to educate my own kids.

Following up on the conversation about AI’s impact on expertise,

Logan Thorneloe

argues that you should never let AI debug for you. The core of his argument is that tedious is not the same as bad. This applies to many other contexts as well.

Replacing all tedious software engineering tasks with AI is a problem. Just because a task is tedious doesn’t mean it’s bad. In software engineering, the tedious tasks are often the ones we learn the most from. The tedium can be a struggle, but it’s the effort required on these tasks that helps us improve.

Lastly, one of my own from the archives. This one is from October of 2023, thinking out loud about how I’d respond to

Marc Andreessen

’s Techno-optimist manifesto. Lots of the same threads I pull on in this piece about the questions we should be asking.

---

The Book Nook

I’ve been enjoying Sprint by

Jake Knapp

. I’m pondering how the framework that Jake writes about here could be mapped into the context of a course. What if we explicitly designed our courses around a series of sprints?

---

The Professor Is In

On Saturday, I had the chance to give a talk as part of a colloquium hosted by the Octet Collaborative at MIT titled “What Will We Become with AI?” Although I could regrettably only participate over Zoom, it was a great day and I learned a lot. In my talk I focused on how I’m thinking about socio-technical thinkers as a key part of helping us navigate the path forward wisely. If you’re interested, you can peruse the slides here.

---

Leisure Line

My colleague brought tangelos from her tree to our department meeting last week and was kind enough to give me the extras to bring home. Super delicious!

---

Still Life

Spotted this flower on a walk back from Scripps.