Photo Shutterstock Kazem Banihashem and Omid Noroozi (Education and Learning Sciences group) did a scientific experiment to compare the feedback quality from peers with the feedback quality generated by ChatGPT on WUR students’ essays. The results of their experiment were recently published in a scientific journal.
‘Feedback is most effective when personalised, but higher education teachers simply do not have enough time to provide detailed, cognitive, constructive, and personalised feedback for all their students, especially in large classes. That is why peer feedback was introduced originally: to replace teacher feedback, enhance interaction and engage students’, Banihashem explains. ‘But even for students, providing qualitative feedback on peers’ essays is challenging because of psychological, emotional, reliability, and cognitive challenges inherent to peer feedback. Recently, artificial intelligence, such as ChatGPT, emerged. There are studies showing that – at some point – the knowledge from ChatGPT is equal to that provided by students. We wanted to explore whether that potential holds in a real-life educational setting.’
AI-generated feedback
‘We gave the free version of ChatGPT the exact same instructions we provided to students for giving feedback on their peers’ essays’, Noroozi explains the experiment. ‘We compared the feedback quality from peers and ChatGPT based on structural, affective, cognitive, and constructive perspectives. ChatGPT was more successful than peers in providing feedback on the structure of the essays. Peers, on the other hand, were able to offer deeper feedback on cognitive aspects of learning, such as problem identification. We conclude that the two can be complementary.’
Banihashem and Noroozi have an extensive research agenda on the use of AI in education. ‘We want to explore the effects of different prompts on the outcomes of ChatGPT’s feedback on these essays. We expect that there will be significant improvements in the essay quality. Still, we’re also curious how students will implement the feedback when they know it comes from ChatGPT, peers or teachers, as they might have different perceptions towards the feedback regarding reliability and trust’, Noroozi says.
Work with AI
‘We want students to collaborate with AI tools such as ChatGPT in a responsible, transparent, and ethical manner’, Noroozi says. ‘I am not in favour of relying entirely on AI or ChatGPT, as it undermines the concept of interactive learning among teachers, students, and peers. However, AI can be used as a complementary tool.’
Do Banihashem and Noroozi stimulate the use of AI in their courses? Noroozi: ‘I am an educational technologist. Of course, I encourage the use of ChatGPT and AI in education, but just the right amount.’ Banihashem agrees. ‘In that way, ChatGPT could increase students’ critical thinking, argumentation and reasoning.’
Don’t forget that providing peer feedback can also contribute to the learning of students. By providing peer feedback, the feedback provider has to verbalise what a text meant to them, and as a consequence they (may) develop a better understanding of what their own texts do to their readers.