Fake Personableness
The race to find generative AI's GUI is on. Here's why we should avoid the temptation to design our tools to imitate persons.
Thank you for being here. As always, these essays are free and publicly available without a paywall. If you can, please consider supporting my writing by becoming a patron via a paid subscription.
Despite all the news around large language models (LLMs) and generative pre-trained transformers (GPTs), the innovation that launched the generative AI revolution was not an algorithmic invention alone. Yes, the performance of GPT-3.5 reached a level of performance that enabled ChatGPT to take off. However, the innovation that made ChatGPT a viral sensation was the interface GPT-3.5 was wrapped in. As we enter the next phase of generative AI development, the focus is shifting from trying to eke out improved performance by throwing more data and compute at the problem. In turn, I expect that we'll see more and more attention turned toward the productization of these tools. As this pivot occurs, we should pay careful attention to the interface.
Nearly two years into the brave new world of ChatGPT in the wild, the power of the interface is clear. When you fire up ChatGPT, you're greeted with a blank window and some helper text in an empty text entry field labeled "Message ChatGPT". Type in a message, hit send, and you'll see some text pretending to explain what's going on in the background. If you're using one of the traditional models like GPT-4, you'll see a little dot that slowly grows and shrinks with the rhythm of an artificial heartbeat as the algorithm is processing. Then, you'll see the result of the computation unravel word by word across the screen. Open up a different model like OpenAI's o1-preview, touted to provide "advanced reasoning," and you'll see a transcript with even more explicitly anthropomorphized language telling you that the model "thought" for 9 seconds accompanied by a brief transcript of its "thought" process.
Some of you may roll your eyes at the scare quotes. It seems that the engineers designing these interfaces don't feel that there is a meaningful distinction between human reasoning and whatever these AI tools are doing. Or maybe they believe it is obvious that there is no ghost in the machine and that using words "thinking" and "reasoning" to describe what is going on is harmless.
It makes no difference to the LLM whether you label its output as "thought" or "reasoning" but it does matter to the humans that interact with it. That's because it is an it. But these labels and the interactions themselves definitely matter to the humans at the keyboard.
We've already seen how this fake personableness can lead to tragic consequences. But this is not the end. Anthropomorphized chatbots are only the beginning. As we continue to develop new interfaces to generative AI, we'll be repeatedly faced with a question: make the tool appear more or less like a person?
More human-sounding interfaces are riddled with potential harms. We've hardly scratched the surface. But what lessons will we learn?
The chatbot is the equivalent of the command line interface. As a 1990s kid, I grew up in the world of the graphical user interface (GUI). Nowadays, except for the small segment of the population that interacts with computers under the hood, most of us can hardly imagine anything but the click-tap-swipe interfaces that are part of the devices we use every day.
At our current crossroads with generative AI, I sense we're on the cusp of a similar fork in the road. Conversation-simulating chatbots are the command line interfaces of generative AI, but the real question is what the GUI will look like. ChatGPT set the chatbot as the default interface. Even now, two years later, it's the decision that most AI companies have chosen. Name a generative AI tool (Claude, Gemini, Midjourney) and the chatbot interface is almost certainly the default mode of interaction.
As the dream of building superhuman machine intelligence on the backs of LLMs reveals itself to be a delusion, attention and capital will pivot. The next object of attention will be more practical applications of the technology we already have. This will require new imagination around the user interface and experience. Without it, we're destined to repeat the same engagement-maximizing decisions that we made with social media.
The race to the interface is heating up. On the one hand, I am excited to move beyond the limitations of chatbot-based interactions. There seem to me to be many ways to leverage the LLM not as an intelligent being of some sort, but as a natural language processor which works with words instead of numbers.
And yet, the opportunity for new modes of engagement with generative AI brings with it a whole slew of new potential hazards. What we will need, both as users and developers, are principles to help guide our design process. Intentionally designing our tools to avoid the appearance of being a person should be foundational.
Avoid fake personableness.
One of the most popular pieces of advice you'll see about how to use generative AI is to "treat it like a person." The shorthand is understandable. These machines don't work anything like a human mind. They are designed to do sophisticated next-token prediction. Regardless of the fundamental difference, the fact that you can just ask a question or describe a task without needing to format it in a specific way is one of the best things about generative AI as compared to other computational systems.
Before LLMs hit the scene, to talk to a computer you needed to learn its language. Now, it’s “learned” ours. But this capability comes with some significant side effects. While the natural language interface with a tool like ChatGPT or Claude is great for lowering the bar for entry, we rarely consider the effect that treating these tools like people has on us. It’s great that we can write code without learning the syntax of a coding language. But when we interact with AI in the same way that we would interact with a colleague, we muddy the water. When we interact with another human, we understand we should treat that person with dignity. We must act in a way that respects and acknowledges their personhood.
If we fail to do this, harm occurs to both people in the relationship. If we treat the person we are interacting with as less than human, we can cause harm to them, both through our words and actions. But just as importantly, our failure to treat others with care also dehumanizes us. It shapes us in ways that distort our view of the world and allow room for our vices to grow.
We already understand the dual-sided nature of this effect in other relationships in our lives. We know that treating an animal with cruelty can bend and deform us. Even treating inanimate physical objects with scorn can create the conditions for anger, bitterness, and scorn to thrive within us. Our actions form us.
These dangers are exacerbated when we interact with generative AI. When the responses of our AI tools make us feel as if there is a person behind them, we are entering dangerous territory. Not because there actually is a conscious being there with any rights that need respecting, but because the way we treat the people and things around us shapes us.
We’ve already had more than enough examples of the ways generative AI can play on our weaknesses. Eliminating the fake personableness would go a long way to helping to guard against these dangers.
Instead of chatbots designed to sound like humans, what if we pursued a different vision? Even if the LLM provides the best responses to our queries when the input to the model is framed as a conversation, this does not require us to expose this deceptive back-and-forth interaction to the user. The use of first-person pronouns is a particularly unnecessary design decision that seems chosen primarily to maximize engagement.
Instead, we should design our generative AI tools with wrappers that help us to guide our patterns of use. What we need is not a general intelligence that can do whatever we want, but a set of generative AI-powered tools that can do specific tasks for us without requiring us to learn a specialized language.
In this light, I'm reminded of my friend and colleague Andy Crouch's framework of the Innovation Bargain that I wrote about a few months ago. The Innovation Bargain reminds us that every new invention answers four prompts:
Now you can…
You’ll no longer have to…
You’ll no longer be able to…
Now you’ll have to…As we think about the existing and emerging interfaces for generative AI, we had better think about the answers those design solutions provide to these questions.
Let's design our tools to be instruments. A technological artifact designed for a specific task instead of a kitchen sink that we can throw at anything. We need generative AI tools that adopt the design philosophy of the Kindle e-reader. The Kindle solves the Innovation Bargain in a way that few other modern technologies do: it is extremely good at what it is designed to do, while simultaneously being extremely poor what it is not designed to do. Want to read a book? Great, the Kindle makes that easy. Want to scroll your social media feeds on the web? It will work in a pinch, but it will in no way be an enjoyable experience.
The race to the interface is on. Let's steer clear of fake personableness.
Got a comment? Join the conversation.
Reading Recommendations
Why a Technocracy Fails Young People by
.We are not digital beings. We are not chatbots, optimized for achievement and sent to conquer this country and then colonize the stars through infinite data accumulation. We are human beings who care deeply about one another because we care about ourselves. Our very existence, as people capable of loving and being loved, is what makes us worthy of the space we occupy, here in this country, on this planet, and on any other planet we may someday find ourselves inhabiting.
Pairs well with
’s piece I shared last week, “For Whom Shall We Build?” (pdf). and have both grappled with the question of whether we should anthropomorphize chatbots. You can read their takes here (Rob) and here (Mike). Here’s the conclusion of Rob’s piece which is closely aligned with my own thoughts on this topic.Instead of treating LLMs like people, let’s approach these tools as potentially useful new cultural technology that has the tricky ability to sound like it knows what it is saying even though it doesn’t. Understood as an interesting but limited form of computational intelligence, an LLM might turn out to do useful things. We will never imagine those uses if all we see is a projection of ourselves.
I enjoyed this piece from
over at where he grapples with construct validity, the necessary preconditions to make sure that we’ve set up our arguments properly. After you read that piece, I also encourage you to check out his piece “So you want to be an LLM-ologist?” which is chock full of great links to chase down.LLM-ologists will need to be comfortable working on questions we don’t know the answer to and which may not even have satisfying answers in the end—something that’s common in science, but which can also be a source of understandable frustration. Ultimately, we need more eyes on the problem: my hope is that this guide will inspire readers to pursue one or multiple of the threads I’ve outlined here, whether that’s asking careful philosophical questions about LLMs, characterizing their internal dynamics, or conducting rigorous experiments to better understand their behavior.
I’m once again sharing this piece on good conflict from
in case you missed it the first time around. Well worth your time.Every conflict has the thing that we fight about and then the thing it's really about underneath. And that’s usually one of four things. It's almost always: respect and recognition; power and control; care and concern; or just flat out stress and overwhelm. But the faster you can identify what the understory is for you and for the other person, the more useful that conflict can be.
The Book Nook
Continuing to slowly work my way through AI Snake Oil by Arvind Narayanan and Sayash Kapoor this week. If you’re looking for a thoughtful, no-nonsense, accessible grounding on AI, it’s a great resource to add to your bookshelf.
The Professor Is In
The 2024 Nelson Lecture Series at Harvey Mudd is now in the books. Last Tuesday was our final talk in the series, a conversation with Dr. Emily Bender, a professor of linguistics at the University of Washington. Our talks followed a decidedly pessimistic trend, beginning with the nearly unbridled optimism of Sal Khan, to
’s more cautious engagement with how generative AI can mask the important work that we should be focusing on as educators, to Emily’s encouragement that “just saying no” is a legitimate answer.I’m thankful for each one of these speakers and the thoughtful conversation that they helped to seed on campus. It was a great opportunity to introduce them to the Harvey Mudd community and engage with each of them.
Leisure Line
If you’re wondering what happens when you let the five-year-old decide how to top the pizza… 😁
Still Life
Meet Pete the Pleco. He’s been a member of our countertop aquarium for some time and is getting more and more adventurous as time goes on. This week I snuck a photo of him while he was out grazing on the bottom of the tank keeping things clean. Normally you can find him chilling upside down attached to the bottom of the filter.
I'm going to play devil's advocate here.
For the last ~15 years, technology has played a role in the steady degradation of human interaction. Social media reduces us to hot takes and memes. We are rewarded by burns and take downs. We're encouraged to focus on building an audience, rather than meaningful interaction. An entire generation now has grown up on this tech. Jon Haidt and others have shown the impact.
While the "humanity" of LLMs is of course an illusion, the illusion could have a positive impact on social development. As you point out, all of our actions influence us. If someone is cruel to an animal, they'll be cruel to humans – they're practicing cruelty. If we're cruel online, we're practicing cruelty, and that will extend to our offline world.
But LLMs are relentlessly polite. They're certainly not constrained to 140 characters! With OpenAI's "advanced voice mode", the AI demonstrates a degree of attention and emotional care which is often lacking from humans, who are absorbed in their own worries.
Of course, a world in which we find more humanity in our interactions with an LLM than actual humans is... problematic. But an optimistic take is that LLMs can counteract the free-fall induced by social networks, and that they can restore the practice of courteous, attentive behavior.
(Probably a pipe dream. But one can hope!)
A critical element here seems this: we must think of the AI as a medium and not as a tool. Here's the distinction: the tool works on a material (say, a typewriter presses keys onto paper into words). The medium is the flexible matter (as, say, a chemical) whose elements allow expression (paint, words, sound). The cello is the tool. The sound is the medium. As an active teacher and a fan of your feed, I would like to engage more on that topic. Any reading recommendations before I do, something that might suit equally students at Harvey Mudd and my college-bound boarding-school students?