Well-timed essay for those of us retooling courses for this fall. I decided last spring that I was going to preserve a long-form research essay assignment in the graduate history of higher education class I will be teaching. I am redesigning the class with more structured group activities during class time and invite the students to experiment with an LLM as an educational tool that can give them feedback on their writing along with peer review and my own reading of their drafts.
I've taught the course three times online, and this is my first in-person teaching since the pandemic. The return to a physical learning space is allowing me push hard into my Deweyan beliefs in student-centered social activities as the path to learning. Teaching undergraduates in the past using those methods, I often received feedback that the approach was uncomfortable and too difficult, something that the literature on this approach bears out....many students don't like it. Online really didn't work either because I couldn't get past how easy it is for students to disconnect when on a screen.
I'm hoping that the shared questions and confusion about the educational value of generative AI tools and the fact my students are mostly master's-level enrolled in a professional higher ed administration program, will mean they will be more open to experiment as well as student-developed and led assignments. The class meets on Friday afternoons, which has limited enrollment (so far 7 students when usually I get 20), which may also help create the dynamic I'm looking for. Plus, I'll get a teaching space suited to in-class activities, which has been a challenge for me in the past. Nothing like fixed seating to add one more challenge to active learning.
Your essay, along with some of Marc Watkins recent essays are going on the first day syllabus for students to better understand where I am coming from when it comes to writing. Thanks, as always, for such a clear articulation of an important educational idea.
Thanks for sharing, Rob. I like your language of "inviting" students to experiment with LLMs. A framing that helps to demonstrate the shared responsibility of learning in the classroom and the agency that students have in their own development.
I look forward to hearing how your class goes this fall. Totally agree that small class size coupled with a flexible physical space provides a lot of potential for active learning to happen!
Learning long-form essay writing in high school played a critical role in preparing me for research. Sure, I could have easily cheated and re-wrote other people's work just as easy as talking to an LLM, but it wouldn't have helped me reach my goals. I agree that in the end, focusing on creating assignments torwards learning goals rather than to block cheating is the way to go.
Also, Needtobreath is my favorite band, such good performances and Christ-centered messages.
I agree. I felt this post concluded without a real conclusion at all. Plus I think the building of character should happen long before a student reached college.
Thanks Elle, I wrote a reply in the thread to Shreeharsh to flesh out some more concrete details that may address some of your comment here.
I also agree that character is being built long before students get to college, but would argue that character is always being built and that the stage of emerging adulthood is an especially formational time in a person's life. I think the college years have great potential for shaping a student's vision of a meaningful life and helping them to build the character traits for them pursue it well.
Agree with the sentiment in theory but this post would have been stronger with a concrete example.
I myself don't feel that threatened because my assignments are not done well by ChatGPT but let's take a colleague's course on Dante's The Divine Comedy. Here, the learning goals of the class--being able to write interpretations of the text--are very much amenable to ChatGPT; the damn thing is stuffed on Dante discourse. He shifted to an exam rather than a paper (fortunately you can test this skill in exams too) but what would you recommend he do?
Fair point, Shreeharsh. I agree that more concrete examples would help to further the dialogue. Perhaps this would be a good topic for next week's post.
Thank you for providing an example to wrestle with. My perspective is that the question of whether an assignment is amenable to ChatGPT is of secondary importance. The first question is whether the task is worth doing on its own merits. This is what I was trying to get at with the marathon/motorcycle example (but am sure I could have articulated it more clearly). In your example, if the learning goal of writing interpretations of Dante is valuable, then we should keep the assignment and better justify it and the learning goals to students (e.g., why an LLM does not eliminate the value in reading and synthesizing information in interpretations). In this frame, the instructor is more like a guide or coach rather than a judge. I think the more we get into this mode with students the better.
There is a second question that also needs to be considered in parallel: does an LLM truly function as a full replacement for the skills we are trying to teach students? If so, then we may need to adapt the learning goals of the assignment in order to respond to and leverage LLMs. My sense is that in most cases, we believe that the sort of work students are doing in your class and in your colleague's class is worth doing not primarily because of the product (the analysis) but because of what students learn through the process (e.g., close reading, careful construction of argument, communication with clarity and nuance, etc.). If these remain valuable skills even in a world with LLMs (and I believe they do), then we should resist the urge to abandon the assignments that we have developed to help students build these skills. We certainly need to be wary of LLMs and adaptations may be warranted, but I'm not sure that moving to an exam from a paper is the right response. These tools may both perform similar functions as an assessment, but they are very different in terms of the type of learning and cognitive labor that they cultivate.
Hi Josh. Thanks for the thoughtful response. I don't think we disagree on the big issues of pedagogy here (what's a worthy learning goal? how does one make that learning goal something students are inspired to do? how does one design interesting assessments (but not necessarily to grade) around satisfying that learning goal? etc.).
I do think we have a fundamentally different view of students and how much instructors can "justify" or "guide" or "coach" students into appreciating the learning goals. I believe that instructors can only do so much to motivate students intrinsically while I think you have much more faith in the power of instructors to do so.
Let's take the question of reading and interpreting and applying Dante. You could say a lot to students about why this is important and they may even agree with you. But there are so many other factors that shape what students do. They may only have half an hour to spend on this assignment. They may have other classes that they are far more interested in. They may only be taking this Dante class to satisfy a requirement. I can think of a thousand different reasons why a student who appreciates the value of the assignment in theory might end up using ChatGPT in practice.
My experience with teaching tells me that while we have to do everything in our power to make our subjects interesting to our students, ultimately, our power is quite limited and extrinsic motivation definitely has a place in the classroom, maybe even a sizable place.
I think my colleague's decision to shift to an exam makes a lot of sense. Exams too can be designed well; I can totally see an open-book, open-notes exam (but no ChatGPT and no internet) that really forces students to answer creative questions.
Several points I'd like to touch on and I completely agree with you.
- Utilizing LLMs for students to shortcut essays seems like nothing more than a new twist on a decades-old problem. Plagiarism or copying other students' answers in any form has always been a problem in schools and using an LLM doesn't seem to be a whole lot different. Students 75 years ago could have (almost as) easily gone to a textbook in a library and copied whole pages to put into their essays. Now we just have LLMs which are only slightly less work. I agree with you that trying to paradigm shift testing due to the introduction of AI seems misguided.
- Changing testing models because of the invention of AI seems like a strange solution and the marathon analogy fits well and again highlights an age-old problem at schools. Someone fit for the marathon wouldn't perform well at a 100 m race. Testing 100 students only on memorization doesn't necessarily highlight their development, but only their ability to memorize which they may or may not ever be good at.
- I believe LLMs have only highlighted a problem that's been building in schools, around the world: reward for doing minimal or no work. Students have leaned on resources to do work for them, and have had little or no consequence from it. They fail exams or assignments but still move on to the next grade. They use LLMs for writing and still pass essay assignments. The culture of focusing on reward for no effort has become a widespread problem that is only getting worse with new technologies increasing convenience and ease of access. Work ethic and patience seem to be missing a lot these days, unfortunately.
Well-timed essay for those of us retooling courses for this fall. I decided last spring that I was going to preserve a long-form research essay assignment in the graduate history of higher education class I will be teaching. I am redesigning the class with more structured group activities during class time and invite the students to experiment with an LLM as an educational tool that can give them feedback on their writing along with peer review and my own reading of their drafts.
I've taught the course three times online, and this is my first in-person teaching since the pandemic. The return to a physical learning space is allowing me push hard into my Deweyan beliefs in student-centered social activities as the path to learning. Teaching undergraduates in the past using those methods, I often received feedback that the approach was uncomfortable and too difficult, something that the literature on this approach bears out....many students don't like it. Online really didn't work either because I couldn't get past how easy it is for students to disconnect when on a screen.
I'm hoping that the shared questions and confusion about the educational value of generative AI tools and the fact my students are mostly master's-level enrolled in a professional higher ed administration program, will mean they will be more open to experiment as well as student-developed and led assignments. The class meets on Friday afternoons, which has limited enrollment (so far 7 students when usually I get 20), which may also help create the dynamic I'm looking for. Plus, I'll get a teaching space suited to in-class activities, which has been a challenge for me in the past. Nothing like fixed seating to add one more challenge to active learning.
Your essay, along with some of Marc Watkins recent essays are going on the first day syllabus for students to better understand where I am coming from when it comes to writing. Thanks, as always, for such a clear articulation of an important educational idea.
Thanks for sharing, Rob. I like your language of "inviting" students to experiment with LLMs. A framing that helps to demonstrate the shared responsibility of learning in the classroom and the agency that students have in their own development.
I look forward to hearing how your class goes this fall. Totally agree that small class size coupled with a flexible physical space provides a lot of potential for active learning to happen!
Learning long-form essay writing in high school played a critical role in preparing me for research. Sure, I could have easily cheated and re-wrote other people's work just as easy as talking to an LLM, but it wouldn't have helped me reach my goals. I agree that in the end, focusing on creating assignments torwards learning goals rather than to block cheating is the way to go.
Also, Needtobreath is my favorite band, such good performances and Christ-centered messages.
I agree. I felt this post concluded without a real conclusion at all. Plus I think the building of character should happen long before a student reached college.
Thanks Elle, I wrote a reply in the thread to Shreeharsh to flesh out some more concrete details that may address some of your comment here.
I also agree that character is being built long before students get to college, but would argue that character is always being built and that the stage of emerging adulthood is an especially formational time in a person's life. I think the college years have great potential for shaping a student's vision of a meaningful life and helping them to build the character traits for them pursue it well.
Agree with the sentiment in theory but this post would have been stronger with a concrete example.
I myself don't feel that threatened because my assignments are not done well by ChatGPT but let's take a colleague's course on Dante's The Divine Comedy. Here, the learning goals of the class--being able to write interpretations of the text--are very much amenable to ChatGPT; the damn thing is stuffed on Dante discourse. He shifted to an exam rather than a paper (fortunately you can test this skill in exams too) but what would you recommend he do?
Fair point, Shreeharsh. I agree that more concrete examples would help to further the dialogue. Perhaps this would be a good topic for next week's post.
Thank you for providing an example to wrestle with. My perspective is that the question of whether an assignment is amenable to ChatGPT is of secondary importance. The first question is whether the task is worth doing on its own merits. This is what I was trying to get at with the marathon/motorcycle example (but am sure I could have articulated it more clearly). In your example, if the learning goal of writing interpretations of Dante is valuable, then we should keep the assignment and better justify it and the learning goals to students (e.g., why an LLM does not eliminate the value in reading and synthesizing information in interpretations). In this frame, the instructor is more like a guide or coach rather than a judge. I think the more we get into this mode with students the better.
There is a second question that also needs to be considered in parallel: does an LLM truly function as a full replacement for the skills we are trying to teach students? If so, then we may need to adapt the learning goals of the assignment in order to respond to and leverage LLMs. My sense is that in most cases, we believe that the sort of work students are doing in your class and in your colleague's class is worth doing not primarily because of the product (the analysis) but because of what students learn through the process (e.g., close reading, careful construction of argument, communication with clarity and nuance, etc.). If these remain valuable skills even in a world with LLMs (and I believe they do), then we should resist the urge to abandon the assignments that we have developed to help students build these skills. We certainly need to be wary of LLMs and adaptations may be warranted, but I'm not sure that moving to an exam from a paper is the right response. These tools may both perform similar functions as an assessment, but they are very different in terms of the type of learning and cognitive labor that they cultivate.
Hi Josh. Thanks for the thoughtful response. I don't think we disagree on the big issues of pedagogy here (what's a worthy learning goal? how does one make that learning goal something students are inspired to do? how does one design interesting assessments (but not necessarily to grade) around satisfying that learning goal? etc.).
I do think we have a fundamentally different view of students and how much instructors can "justify" or "guide" or "coach" students into appreciating the learning goals. I believe that instructors can only do so much to motivate students intrinsically while I think you have much more faith in the power of instructors to do so.
Let's take the question of reading and interpreting and applying Dante. You could say a lot to students about why this is important and they may even agree with you. But there are so many other factors that shape what students do. They may only have half an hour to spend on this assignment. They may have other classes that they are far more interested in. They may only be taking this Dante class to satisfy a requirement. I can think of a thousand different reasons why a student who appreciates the value of the assignment in theory might end up using ChatGPT in practice.
My experience with teaching tells me that while we have to do everything in our power to make our subjects interesting to our students, ultimately, our power is quite limited and extrinsic motivation definitely has a place in the classroom, maybe even a sizable place.
I think my colleague's decision to shift to an exam makes a lot of sense. Exams too can be designed well; I can totally see an open-book, open-notes exam (but no ChatGPT and no internet) that really forces students to answer creative questions.
But I think we agree more than we disagree.
Several points I'd like to touch on and I completely agree with you.
- Utilizing LLMs for students to shortcut essays seems like nothing more than a new twist on a decades-old problem. Plagiarism or copying other students' answers in any form has always been a problem in schools and using an LLM doesn't seem to be a whole lot different. Students 75 years ago could have (almost as) easily gone to a textbook in a library and copied whole pages to put into their essays. Now we just have LLMs which are only slightly less work. I agree with you that trying to paradigm shift testing due to the introduction of AI seems misguided.
- Changing testing models because of the invention of AI seems like a strange solution and the marathon analogy fits well and again highlights an age-old problem at schools. Someone fit for the marathon wouldn't perform well at a 100 m race. Testing 100 students only on memorization doesn't necessarily highlight their development, but only their ability to memorize which they may or may not ever be good at.
- I believe LLMs have only highlighted a problem that's been building in schools, around the world: reward for doing minimal or no work. Students have leaned on resources to do work for them, and have had little or no consequence from it. They fail exams or assignments but still move on to the next grade. They use LLMs for writing and still pass essay assignments. The culture of focusing on reward for no effort has become a widespread problem that is only getting worse with new technologies increasing convenience and ease of access. Work ethic and patience seem to be missing a lot these days, unfortunately.