Lesson Trial Runs

Last updated on 2024-01-30 | Edit this page

Teaching a lesson for the first time is an important intermediate step in the lesson development process. During the break between the first and second parts of this training, we ask trainees to teach part of the lesson they have been designing to a real audience. The purpose of this task is to gather feedback at an early stage of the development process, and to reflect on how well the tested lesson content meets the objectives that we defined for it.

On this page, we provide details of the task and some guidance on what feedback/information to collect while testing out the lesson content.

What should I teach?

During the first stages of the training, you will have completed the early stages of design and development of a episode of your new lesson. After completing the exercises in the training sessions, you should have identified a target audience for the lesson, defined objectives for the lesson and for a particular episode within it, designed exercises to assess learner progress towards the episode objectives, and outlined a narrative to lead learners through that episode. You should use the trial run as an opportunity to teach this episode of your lesson to a real audience. Note that it will probably be helpful to spend some time writing some of the supporting information for your lesson for this trial run. For example, if you did not have time to finish writing the setup instructions for your lesson in the learners/setup.md file, you should try to complete these and share them with your learners before the trial run.

For groups of collaborators

During the training, you should have been working on consecutive episodes in your lesson. Ideally, use the trail run as an opportunity for you to test the lesson content together, with each trainee teaching the episode they have been working on during the training in a single trial run event. This will give you an opportunity to observe each other teaching, to help each other gather feedback and information to help improve the lesson (see below), and to identify any gaps or overlaps that become apparent when moving from one episode to the next.

If you cannot all teach an episode, make sure that every collaborator can take an active role in your Trial Run, for example by make those who will not teach responsible for taking notes on timings, questions asked by learners, technical difficulties, etc.

For Lesson Developers working alone

We ask you to collect a lot of information during the trial run (see below), and it will be difficult to do this while also devoting yourself to delivering the lesson. For that reason, we recommend that you try to find someone who can attend your trial run as an observer, to take notes and give you feedback from their perspective. Ideally, this would be someone who you work with and who is already familiar with the topic that will be taught in your lesson episode. That will make it easier for them to provide their own insights, and to properly capture details in the notes they take during the trial run. If you are unable to recruit a colleague or peer to do this for your trial run, please discuss it with your trainers and/or send an email to curriculum@carpentries.org to request support.

What information should be collected from the Trial Run?

During the trial run, it will be helpful to take notes to answer the following questions:

How long did it take to teach the episode?
How long did learners need to complete each exercise?
What questions did learners have during the trial run?
What technical difficulties were encountered during the trial run?
What parts of the lesson appeared to be confusing for learners?

At the end of the trial run, make sure to capture some feedback from the learners. You can use the methods described in Preparing to teach to gather feedback, e.g.

Minute cards
One up, one down
Ask learners about the exercises in your episode

As soon as possible after the trial run has finished - ideally immediately afterwards - take some time to reflect on the following questions, based on the feedback collected and from your own perspective as the instructor and author of the lesson content. Make some notes on your answers to these questions. You will need to return to these notes when you join the second part of the training.

What worked well both in terms of content and delivery?
What did not work as well?
How close were your time estimates to the actual time needed to teach the material and finish exercises?
How did the audience perceive the difficulty level of the material?
What will you do differently next time?
What will you change in the material you taught?
What will you change in the way you collect feedback in future pilots?

When you have completed the trial run and made these notes, send them to your trainers using the contact details they will share during the training.

Who should the learners be for a trial run?

You can choose the format and audience for your trial run. It could:

take place online or in-person.
be a private session attended by invitation only, or open to external participants.
be delivered to members of your own network, community, or institution.
be advertised on The Carpentries channels and delivered to members of the wider community.

Feedback and experience collected from testing lesson material like this will be most useful if the audience taught closely matches the intended audience for the lesson itself. However this is often not easy for a short trial run, especially if your chosen episode does not appear early in the lesson (as audience members will not have benefited from learning the previous episodes first). In this case, try to ensure that members of the audience are briefed on what kind of feedback to give, e.g. by providing context about what would have been covered in the earlier sections of a full workshop teaching the whole lesson. If you are teaching to people who are already familiar with the lesson topic, i.e. who are not representative of the actual target audience for your lesson, encourage them to focus their feedback on points that would be most important to someone who is learning the skills and concepts in the lesson for the first time.

Where can I go for help?

The Carpentries Curriculum Team can provide support for trial runs. For example, by providing access to an account with a paid Zoom license for an online trial run, by helping advertise the session to The Carpentries (sub)communities, by listing sessions on The Carpentries Community Calendar, etc. Contact the team on curriculum@carpentries.org if you would like to request assistance like this.