Great piece, Josh. So glad to see you featured in that Substack Reads, guest written by the great Henry Oliver. I just picked up Ethan Mollick's new book and I'm coming at it with questions about his use of metaphors drawn from magic...centaurs, grimoires, and other images drawn from fantasy. I tend to think language scifi and fantasy mostly interfere with our understanding of what foundation models are doing, but I also find his explorations compelling in the sense that it can give us a handle on phenomena that are very hard to describe because they are so new. Your pointing to the role illusion plays in the way entertainers who call themselves magicians is very helpful in that context.
Thanks Rob. I am looking forward to reading Ethan Mollick's new book too.
I like how you highlight the work that metaphors and language shapes our understanding of these tools and what they are doing. In this space it feels particularly important that we are careful to think through these metaphors and to understand their strengths, limits, and potential blind spots. I'll be curious to hear your reflections as you approach Mollick's work with a critical eye toward his use of mythological archetypes.
This was a really interesting read for me, thanks Josh. The most impressive thing for me I’ve noticed so far with writers on Substack is their ability to make a connection between simple topics and make profound connections. Not coming up with the most groundbreaking new theory, but taking the most mundane concepts and reframing them in a way to look differently and see the brilliance in something fairly normal to us, such as shuffling cards.
Thanks Jacob. Yes, I love to look for these examples in my writing—trying to synthesize and make the connections between big ideas and concrete examples. I'm glad that this resonated with you!
Josh - great post - to take your analogy one step further, we need to go behind the hype, to use critical thinking and science to really understand what AI LLM's are doing, in real time.
One of the ways we do this in other system is through measurement and observation. What are you seeing in terms of how to measure AI's accuracy and output to cut through the illusory hype?
Thanks David. I agree that the transparency and explainability in AI is going to be increasingly important to help put these tools in the proper view.
It's a good question about how to effectively cut through the hype. I think what is most interesting to me is the efforts to help provide more sources to explain where particular text is coming from to help trace references. But I do think there are some fundamental technical challenges connected to solving this problem and I'm not sure if the GPT foundation model will be able to address this on its own.
For me the biggest thing to begin with is to keep a healthy sense of skepticism about the outputs that we get from these machines and to always be interrogating and thinking critically about the outputs.
Late to the chase, but I want to reaffirm cupcakes and spring flowers for the win! (Both, sadly, in short supply here, at the moment.)
There are always so many threads to pick up and follow in your posts, Josh; thank you for continually sending so much out into the world. Lots to think about; you are a good catalyst.
Thank you, too, for sharing simply and honestly about your religious faith - I appreciate it’s not easy to talk about those topics alongside STEM topics.
I think your choice of illusion as a keyword is quite important as we try and take in and understand AI, with new developments and commentary available every hour of every day, it seems. And, of course, illusion can double-back and converse with religious faith — something about “the evidence of things not seen” — illusion or reality?
“What does it mean to be human?” — At the end of the day, though, I feel all the ponderings and doubts and uncertainties and so on, do come down to this question you posed. Through whatever lenses we use, and whatever bewildering combinations of right and wrong actions we perform, everyone has to come up with their own response to this most personal and universal of mysteries — “What does it mean to be human?”
Glad and grateful to be along with you on this journey (even if the cupcakes are 3000 miles away, lol).
Great piece, Josh. So glad to see you featured in that Substack Reads, guest written by the great Henry Oliver. I just picked up Ethan Mollick's new book and I'm coming at it with questions about his use of metaphors drawn from magic...centaurs, grimoires, and other images drawn from fantasy. I tend to think language scifi and fantasy mostly interfere with our understanding of what foundation models are doing, but I also find his explorations compelling in the sense that it can give us a handle on phenomena that are very hard to describe because they are so new. Your pointing to the role illusion plays in the way entertainers who call themselves magicians is very helpful in that context.
Thanks Rob. I am looking forward to reading Ethan Mollick's new book too.
I like how you highlight the work that metaphors and language shapes our understanding of these tools and what they are doing. In this space it feels particularly important that we are careful to think through these metaphors and to understand their strengths, limits, and potential blind spots. I'll be curious to hear your reflections as you approach Mollick's work with a critical eye toward his use of mythological archetypes.
This was a really interesting read for me, thanks Josh. The most impressive thing for me I’ve noticed so far with writers on Substack is their ability to make a connection between simple topics and make profound connections. Not coming up with the most groundbreaking new theory, but taking the most mundane concepts and reframing them in a way to look differently and see the brilliance in something fairly normal to us, such as shuffling cards.
Thanks Jacob. Yes, I love to look for these examples in my writing—trying to synthesize and make the connections between big ideas and concrete examples. I'm glad that this resonated with you!
Thanks for the shoutout!
Josh - great post - to take your analogy one step further, we need to go behind the hype, to use critical thinking and science to really understand what AI LLM's are doing, in real time.
One of the ways we do this in other system is through measurement and observation. What are you seeing in terms of how to measure AI's accuracy and output to cut through the illusory hype?
Thanks David. I agree that the transparency and explainability in AI is going to be increasingly important to help put these tools in the proper view.
It's a good question about how to effectively cut through the hype. I think what is most interesting to me is the efforts to help provide more sources to explain where particular text is coming from to help trace references. But I do think there are some fundamental technical challenges connected to solving this problem and I'm not sure if the GPT foundation model will be able to address this on its own.
For me the biggest thing to begin with is to keep a healthy sense of skepticism about the outputs that we get from these machines and to always be interrogating and thinking critically about the outputs.
Late to the chase, but I want to reaffirm cupcakes and spring flowers for the win! (Both, sadly, in short supply here, at the moment.)
There are always so many threads to pick up and follow in your posts, Josh; thank you for continually sending so much out into the world. Lots to think about; you are a good catalyst.
Thank you, too, for sharing simply and honestly about your religious faith - I appreciate it’s not easy to talk about those topics alongside STEM topics.
I think your choice of illusion as a keyword is quite important as we try and take in and understand AI, with new developments and commentary available every hour of every day, it seems. And, of course, illusion can double-back and converse with religious faith — something about “the evidence of things not seen” — illusion or reality?
“What does it mean to be human?” — At the end of the day, though, I feel all the ponderings and doubts and uncertainties and so on, do come down to this question you posed. Through whatever lenses we use, and whatever bewildering combinations of right and wrong actions we perform, everyone has to come up with their own response to this most personal and universal of mysteries — “What does it mean to be human?”
Glad and grateful to be along with you on this journey (even if the cupcakes are 3000 miles away, lol).