Stay on Target

Last updated on 2023-07-10 | Edit this page

Estimated time 65 minutes

Overview

Questions

  • How can you measure learners’ progress towards your lesson objectives?
  • Why is it important to identify misconceptions as early as possible?
  • Why should we create assessments before we have written the explanatory content of our lesson?

Objectives

After completing this episode, participants should be able to…

  • Explain what is meant by the intended and attained curriculum of a lesson.
  • Describe the importance of regular assessment while a lesson is being taught.
  • Design assessments to identify the misconceptions learners might have during your lesson.

As we have seen previously, defining objectives for a lesson (or a teaching episode) can help to focus your content on the most important learning outcomes and outline the scope of your lesson project. With learning objectives you have defined your intended (or planned) curriculum - what you set out to accomplish with your lesson. The skills and knowledge your learners actually leave with and can demonstrate after the workshop having followed the implemented curriculum in the classroom is called the attained curriculum. Implemented curriculum includes the concepts, relationships and skills that are explicitly covered in your lesson and translated into classroom practices - i.e. the teaching and learning activities that should lead to your learners attaining knowledge.

Hidden Curriculum

There is one more curriculum at play here - hidden curriculum includes all unintended curricular effects that influence learning but are not explicitly addressed in the lesson content - e.g. (unofficial) social and cultural norms, behaviours and values that are transferred by instructors to learners, often unconsciously. These are all the additional things that people learn about the topic from the way your lesson is taught, rather than from its official content. You can use Instructor Notes (covered in more detail in “Preparing to Teach” episode) to guide instructors on some of the aspects of the hidden curriculum, e.g. to encourage them to follow certain tried and tested practices.

The goal of the remaining steps of lesson development is to ensure that the attained curriculum matches the intended curriculum as closely as possible. To do so, you need to develop assessments to ensure progression towards your learning outcomes.

Assessments

In order to measure progress and evaluate if and what learning occurred - we use various types of assessments:

  • summative assessments - used to verify whether learners achieved the stated learning objectives after instruction.
  • formative assessments - used to detect changes in learner performance during instruction, to provide feedback and insight into the learners’ developing mental models of the topic taught and to identify any old or developing misconceptions.

Summative assessments sum up what learning has been achieved after training (e.g. via exams). They give valuable data about learning attainment by individuals and entire cohorts but are not used to guide further progress. They may not be as suitable for short courses, but may be necessary for those that give marks/grades or certificates of completion.

Formative assessments are applied throughout a course and with several different purposes: they provide a way to move new information from working memory to long-term memory; they can inform instructors’ decisions about how to modify instruction to better promote learning; they also inform learners about changes they may need to make to improve their learning. Ideally, they should be used often (e.g. after every 15-20 minutes of teaching), providing opportunities to instructors to change pace and refocus learners’ attention. For short courses, formative assessments are usually more valuable and easier to implement in practice than summative assessments - they need not be complex or time-consuming, just informative enough about learning for both instructors and learners.

The most effective way to test learner understanding is to do such assessments in class - they engage all learners and allow instructors to check learners’ confidence with the content and its delivery, can help you deal with any potential misunderstandings as soon as they arise, and maximise the value of workshop for everyone. Such formative assessments also help with metacognition - the awareness a learner has that they are succeeding in learning something new.

Any instructional tool that generates feedback and is used in a formative way to check for learners’ understanding can be described as “formative assessment”. For example,

  • reflection at the end of a session to help process learning - e.g. asking learners to write down things they learned, things they want to know more about and any questions they still have
  • concept maps and diagrams - asking learners to reflect by drawing/labeling a concept map/diagram or writing down a list of new concepts and skills they’ve learned and (optionally) how they relate to one another or connect with previous knowledge
  • checking in - gauging learners’ satisfaction and understanding using agreed signals (e.g. raising different coloured post-it/sticky notes or Zoom reactions to indicate that the pace is too fast/slow, that they completed/have not completed an exercise).

Many other formative assessment tools can be found in Briggs’ list of “21 ways to check for student understanding” or Edutopia’s “56 Examples of Formative Assessment”.

Exercises are one important type of formative assessment. We will now have a look into exercises that perform misconception checks and ask students diagnostic questions; we will cover a few other types of exercises that help with retaining new knowledge in one of the later episodes.

Detecting Misconceptions

Detecting and correcting misconceptions and fixing learners’ incorrect/broken mental models is as important as presenting your learners with new knowledge and correct information. Why is it important to identify misconceptions as early as possible? When mental models are broken, learning can occur slower than you might expect1. The longer a prior incorrect model is in use, and the more extensively it has to be “unlearned”, the more it can actively interfere with the incorporation of the new correct knowledge (since it will contradict the misconceptions already present in the mental model).

Exercise: misconceptions (5 minutes)

What are the common misconceptions learners can have about the topic of your lesson? How might you identify that misconception in your learners while they follow your lesson? Share your answer in the collaborative notes document.

Hint: Try thinking about related or common tools the learners might know and how applying that prior knowledge might lead to a misconception with the topic you are teaching.

Multiple choice question (MCQ) exercises are types of a formative assessment that can help you target anticipated misconceptions. When designed carefully, each incorrect answer in a MCQ will have diagnostic power and provide valuable insight into how a mental model is broken. For example, suppose we are teaching children multi-digit addition. An example of a well-designed MCQ (borrowed from The Carpentries Instructor Training) in this case could be:

Q: What is 27 + 15?

  1. 42
  2. 32
  3. 312
  4. 33

The correct answer is 42, but each of the other answers provides a valuable insight:

  1. they do not understand the concept of a carry and are throwing it away completely
  2. they understand the concept of a carry and know that they cannot just discard the carried ‘1’, but do not understand that it is actually a ten and needs to be added into the next column - they are treating each column of numbers as unconnected to its neighbours.
  3. they understand that they need to carry ‘1’ but are adding it to the wrong column.

Their diagnostic power means that each of the wrong choices helps an instructor figure out precisely what misconceptions learners had adopted and in which ways their mental models are broken. As a result the instructor may decide to review a particular concept or change the pace of instruction. At the same time, learners get feedback about what they have misunderstood and what they need to focus their study efforts on - we call this guided practice.

When designing a lesson, for diagnostic assessments you can think about problems or questions from previous training events and what people struggled with, or think about your own misconceptions in the past or ask colleagues about their experiences. You should aim to create all your assessments before you have written the explanatory content of your lesson (recall Nicholl’s backward design). These assessments will guide your lesson design process by knowing exactly which knowledge you’d expect from your learners at any point in the lesson.

Exercise: designing a diagnostic exercise (20 minutes)

Create a multiple choice question (MCQ) that could be used in your lesson, to detect the misconception you identified above. As well as the correct answer, include 1-3 answer options that are not obviously incorrect (plausible distractors) and have diagnostic power i.e. each incorrect answer helps you pinpoint the exact misconception carried by the learner.

Share your MCQ in the collaborative notes document.

Exercise: reviewing formative assessments (10 minutes)

(this exercise will only work if participants have sufficient knowledge of their partner’s topic)

The Trainers will group you into pairs.

Review the MCQ designed by your partner. When providing feedback, try to answer the following questions:

  • Is the question clear and easy to understand? Could the wording be improved in some way?
  • Are the incorrect answers to the MCQ plausible distractors?
  • Do the incorrect answers provide diagnostic power, to help an Instructor identify the misconception the learner has?
  • Are there any incorrect answers missing i.e. are there other misconceptions that could be detected with this MCQ?

Share your feedback in the collaborative notes document.

Keypoints

  • The goal of lesson development is to ensure that the attained curriculum matches the intended curriculum as closely as possible.
  • Assessments are a way to determine whether the objectives you defined for the lesson have been reached.
  • Formative assessment happens during teaching and provides feedback both to an instructor and a learner - about progress and whether learning of new concepts occurred but also about any misunderstandings and misconceptions which can hinder further learning.
  • It is important to detect misconceptions as early as possible and formative assessments (such as multiple choice questions) can help us with this.

  1. See chapter 1, How Does Students’ Prior Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎