The User Interface Really Matters

The flexibility of LLMs requires that we be especially thoughtful about the user interface decisions we make in order for them to be fruitful in education

Thank you for being here. As always, these essays are free and publicly available without a paywall. If my writing is valuable to you, please share it with a friend or support me with a paid subscription.

---

Image source.

If you pay close enough attention, there’s an interesting feature of many of the ways we’re seeing AI used in education today. The same tool that can be used by one person to help them learn more effectively can be used by another to completely undermine their learning altogether. The capability of the AI tool is only part of the story. What’s more important are the user interface and experience design decisions that guide a user’s interaction.

The Challenge of Flexibility

This phenomenon is directly connected to the flexibility of particular technologies. Textbooks are relatively inflexible technologies. You can change the order in which you read the book, but the words on the page and the topics in the book never change. The web, as mediated through the Google search bar, adds a new layer of flexibility. Now you can decide on the filter, choosing specific keywords to include or exclude, compiling your own set of sources. Unlike the textbook, you are no longer relying on the author’s curating wisdom, for better and for worse.

The large language model takes this one step further, allowing you to interact with the contents of the training data—all of the text in human history, to a close approximation, copyrighted materials included—through a prompt. The prompt lets you shake a sophisticated and tremendously expensive magic eight ball (i.e., an LLM) in a particular way. In response, you get out the continuation of the prompt, something that is a syntactically coherent interpolation of the information in the training data. The result returned to you is statistically likely to be located near your prompt in some high-dimensional vector space.

The flexibility of this kind of tool is a double-edged sword in educational situations. On the one hand, it provides a very interesting way of interacting with data in a flexible manner. It’s easier than ever to start where you are. You just need to describe what you know and what you’re curious about. No need to look in the index at the back of the book or even come up with a few relevant keywords for a web search. Just type something in and you’ll start to weave your way through the data. If you want to feel particularly special, imagine you are Paul Atreides heroically riding on top of a sandworm as your prompt finds its continuation. (A sandworm feels like an appropriate metaphor for large language models in more ways than one. I digress.)

Powerful Tool or Perilous Threat

What is making education very confusing in our current moment is that some people see the sandworm as a powerful tool to be harnessed, while others see it as a terrifying beast that will end your life in an instant. What’s even more complicated is that they’re both right.

While there are many ways that LLMs have the potential to make education better and help students to learn more effectively, there are just as many ways (and maybe more) for it to completely undermine the entire educational enterprise. I

t can be used to help students challenge their ideas and brainstorm more divergently, but it can also replace the brainstorming process entirely. It can help surface more relevant resources to use in analyzing and making an argument, but it can also synthesize them without requiring any thought or engagement on the part of the author.

In educational spaces, the flexibility of LLMs often turns out to be more of a bug than a feature. There is a reason, it turns out, that most textbooks are structured in a particular order. Fortunately, this is not the only way that we can interact with AI. A much better way to harness the power of LLMs while mitigating the potential for them to lead us astray is to build AI-powered tools with specific, narrow use cases and well-defined user interfaces that support those goals. It is still early, but there are more and more examples of this kind emerging.

Take Lex, for example. It’s a collaborative word processor like Google Docs, with AI-enabled editing features. Yes, it has a chat sidebar that you can use to chat with your document and ask for particular points of feedback, but the most interesting feature relevant to this conversation is Checks.

Checks are customizable LLM prompts you can use in the interface to suggest edits to your writing. For example, you can use a check for feedback on your grammar.¹ Similarly, you might use a check to flag clichés.²

But even more important than the feature itself is the way that it is integrated within the document. When running checks with Lex, you stay in the document with all the suggested edits made in-line as suggestions. Then you can review and accept the changes if you agree with them or reject them if you don’t. Perhaps a little cliché every now and again isn’t so bad.

The specific limitations of this sort of interface provide opportunities that you might use to offer your students a structured way to incorporate AI feedback. For example, you could consider providing a series of specific prompts for students to use as checks for their writing to help them flag particular high-level feedback that you would commonly give. Just like a spell checker is a tool that can help make better use of human expertise and feedback on a piece of writing, these kinds of tools can help a writer begin to think through slightly higher-level stylistic choices that might otherwise take precedence over broader stylistic or content goals. The key feature is that this design leverages the LLM, but does it in such a way as to keep the writer in the decision-making seat.

In recent talks about AI, I’ve been fond of returning to the Cookies story from Frog and Toad that I previously wrote about in The Cookie Box Principle. The punchline, for those of you who might not remember, is that we should be aware of the limits of our willpower and think carefully about the environment surrounding us.

As we grapple with AI and try to figure out the ways that it might be leading us astray, we should remember that the user interface design decisions of the AI-powered tools we use have a significant impact on how they enable us to engage with the work we are doing and its intended goal. It is wise to create specific guardrails for the AI tools that we encourage our students to use. We should take a cue from applications like Lex to remind us that the popular chatbot interface need not be the only way to interact with an LLM.

Got a thought? Leave a comment below.

Reading Recommendations

The piece to read this week is Derek Thompson’s essay about the greatest challenge that LLMs pose to us. Not that superhuman intelligence will overtake us, but that we’ll outsource our thinking to the existing subhuman intelligence.

He ends with a good suggestion too:

So what should our children study in an age of thinking machines? While I don’t know what field any particular student should major in, I do feel strongly about what skill they should value: It’s the very same skill that I see in decline. It’s the patience to read long and complex texts; to hold conflicting ideas in our heads and enjoy their dissonance; to engage in hand-to-hand combat at the sentence level within a piece of writing — and to value these things at a time when valuing them is a choice, because video entertainment is replacing reading and ChatGPT essays are replacing writing. As AI becomes abundant, there is a clear and present threat that deep human thinking will become scarce.

Timothy Crouch with a thoughtful take on Paul Kingsnorth latest book and whether machine is better understood as mammon.

A really interesting thread from Brad DeLong and Alex Tolley on intelligence, LLMs, and underlying world models.

I haven’t yet picked up the book that everyone in my feed seems to be reviewing these days (If Anyone Builds It, Everyone Dies), but I appreciated this review from Timothy B. Lee.

He breaks their argument for existential risk into three main pieces.

1. Humans are on a path to develop AI systems with superhuman intelligence.

2. These systems will gain a lot of power over the physical world.

3. We don’t know how to ensure these systems use their power for good rather than evil.

Timothy suspects that point 2 is the linchpin.

I think the weakest link in Yudkowsky and Soares’s argument is actually the second claim: that an AI system with superhuman intelligence would become so powerful it could kill everyone. I have no doubt that AI will give people new capabilities and solve long-standing problems. But I think the authors wildly overestimate how transformational the technology will be—and dramatically underestimate how easy it will be for humans to maintain control.

If “power over the physical world” must come in the form of fully integrated humanoid robots, I think there is still a way to go (although perhaps not quite as much as we may think). But this is not the only (or easiest) way to gain power in the physical world. Given the power of LLMs to masquerade as other conscious beings, I’m much more concerned about malicious actors using LLMs to extend and manipulate other humans in the real world, absent any robotics revolution.

---

The Book Nook

Another week, another Trueblood. This week, I’m reading my third Trueblood book, The Predicament of Modern Man. This is perhaps his most well-known volume, offering a diagnosis of what ails us in the wake of World War II.

---

The Professor Is In

Earlier this year, I had the chance to spend some time with the Board of Trustees of Taylor University during their destination board meeting in Waco, Texas. A few weeks ago, I got an email from Taylor’s Dean of Faculty development, asking whether I happened to be in the Midwest anytime soon. Glad I managed to make the quick day trip down from South Bend to meet with some of the campus community at Taylor and dialogue with them about how they are thinking through AI on their campus and the role it should play in a Christian college.

---

Leisure Line

After the conference activities at Notre Dame wrapped up last Tuesday evening, I went for a walk around campus and happened to catch the last half hour or so of their soccer match against Wright State. It was 3-0 when I arrived, and Notre Dame managed to hang on 3-2, despite some late efforts from the Raiders.

---

Still Life

Another trip and another Tesla rental. These cars are just fun to drive. I’m also counting it a success that I managed not to get stranded in a cornfield in Indiana, given the slightly sparse charging infrastructure on my trip to Taylor!

1. Lex Check Grammar Prompt: “You are an editor that will fix any clear spelling or grammatical errors in the user-provided text, otherwise leave the original text unchanged. Do not reply, simply correct the message.”

2. Lex Check Cliché Prompt: Remove any obvious clichés from the user-provided text, otherwise, leave the original text unchanged. For example, you should replace phrases like “gut wrenching,” “circle back,” “at the end of the day,” etc. But you should not replace anything that is not obviously a cliché.”).