Building Skill with Feedback

Overview

Questions

• How might learners’ use of genAI tools impact the pathways and mechanisms of feedback in a workshop?

Objectives

• Identify possible impacts of genAI chatbot use on the established feedback mechanisms in a workshop.
• Discuss how Instructors can encourage learners to continue to provide feedback in a workshop while interacting with a chatbot.

Challenge

Feedback Mechanisms

What is one way that a learner receives feedback on their work or understanding during a workshop? How might the learner’s use of a chatbot disrupt that feedback mechanism?

What is one way that an Instructor receives feedback on learners’ understanding during a workshop? How might the learner’s use of a chatbot disrupt that feedback mechanism?

Show me the solution

Feedback mechanisms for learners include:

• Exercises and worked solutions.
• Comments from Instructors, Helpers, and other workshop participants.
• Their impressions of their own ability to keep up with the pace of instruction, understand what is being taught, etc.

Feedback mechanisms for Instructors include:

• Sticky note feedback.
• Facial expressions and demeanors of learners.
• Results of exercises and other assessments.
• Questions asked and explanations given by learners.

One challenge that learners’ use of genAI tools presents to a workshop is that it removes a crucial pathway of feedback to the Instructor. If a learner turns to a chatbot for help before they turn to a helper or Instructor, the Instructor will never know what question they had, what they were finding difficult, or where they were stuck. This means the Instructor cannot identify trends in what the class is finding difficult and misses the opportunity to clarify the key points.

This is another reason why it is so important to create and maintain a positive learning environment in a workshop. If Instructors and helpers appear approachable to learners, encouraging them to speak up when they are stuck and to share what they are getting from genAI tools if/when they use them, these feedback pathways are more likely to stay open. Nevertheless, you and your helpers may make all of these efforts and still observe learners using a chatbot before talking to a human.

Discussion

How Can We Encourage Feedback?

How could Instructors encourage learners to speak to them or a helper before resorting to a chatbot? What strategies could Instructors implement in a workshop to encourage learners to be open about when and how they are using genAI while they learn? Can you identify any ways that learners’ use of genAI could be harnessed as a teaching opportunity in a workshop?

Strategies that you could try to encourage learners to ask questions to helpers and Instructors rather than a chatbot during a workshop include:

• Explain the importance of questions and discussion feedback to you, and how that feedback influences your approach throughout the workshop. (Refer to Setting Expectations at a Workshop below.)
• Use pair programming and small group discussion for more exercises. These formats will encourage discussion among learners and may make them hesitate and ask one another when they are unsure before they refer to a chatbot.

The availability of answers from a chatbot means that it is more important than ever for us to create and maintain a positive learning environment in our workshops. If learners perceive the Instructors and helpers to be patient, friendly, and approachable (as well as knowledgable and capable of providing accessible answers), they will be more likely to turn to them when they get stuck.

The way that you acknowledge and respond to your own mistakes can be a model of behaviour for learners to follow. The fallibility of the Instructor can be inspiring to learners who may be reluctant to acknowledge when they do not know or understand something they feel that they should.

Callout

Setting Expectations at a Workshop

Address the use of chatbots directly and early in your workshops, and set expectations with your learners about how to make the most of the workshop. Set aside a few minutes during the workshop introduction to cover some or all of the following points:

• Summarise the key points from Instructor Training about how learning works, and use these to discourage the use of chatbots during the workshop.
  • Expertise is built through guided practice, and exercise must be challenging to be useful.
  • Instructors will manage learners’ cognitive load throughout the workshop: the order and pacing is deliberate to help learners build a mental model so that they can continue to learn after they leave.
  • Motivation is essential for learning: encourage learners to identify and connect with their motivation for joining the workshop.
  • Feedback (to the learner and the Instructor) is an essential element of the workshop to ensure learning.
• Explain that the benefit that learners get from the workshop will be proportional to the effort they put into it: the more they apply themselves to the tasks and discussions in the workshop, the more they will develop their own ability, independent of any technological aids.
• Emphasise that questions are welcome and even encouraged. Introduce the helpers and co-Instructors and try to provide some details that will encourage learners to refer to them when they need. Explain how questions asked by one learner are often shared by others, and that the answers will probably benefit everyone.
• Explain the value of learning the basics in the workshop to learners who are keen to start using AI tools to generate their code: the foundational mental model that they will acquire by engaging with the workshop will help them make more effective use of AI assistance afterwards.
• The key takeaways from the Stromberg et al paper make a compelling case for learners not to use AI to help them complete exercises.
• Combine this overview of your expectations of learners with a discussion of what they can expect from you in return.

Key Points

• It can be difficut to gather helpful feedback if learners are using a chatbot instead of requesting assistance from Instructors and helpers.
• Being explicit about your expectations of learners and they ways they should/should not use AI tools in the workshop.