After completing this episode, participants should be able to…
Explain how trainers and participants will interact throughout the
workshop.
Summarise the main skills that will be taught in this workshop.
You might like to make a copy of the template
CodiMD notetaking page and share your link with trainees so that you
can take notes together during the training.
Using correct names and pronouns (e.g. “she/her”) is important to
setting a tone of respect. Learning these is hard to do quickly, so we
recommend displaying it prominently during the workshop.
In an online workshop, give everyone a moment to update their display
name to reflect how they would like to be addressed.
At an in-person event, we recommend supplying name tags and markers,
or using plain paper to create table-displayed name placards.
Note that pronouns are personal and some participants might prefer
not to share them. Do not force people to share their pronouns.
Before The Training Begins
Getting to Know Each Other
If the Trainer has chosen an icebreaker
question, participate by writing your answers in the shared document
for the workshop.
Code of Conduct
To make clear what is expected, everyone participating in The
Carpentries activities is required to abide by our Code of Conduct.
Any form of behaviour to exclude, intimidate, or cause discomfort is a
violation of the Code of Conduct. In order to foster a positive and
professional learning environment we encourage you to:
Use welcoming and inclusive language
Be respectful of different viewpoints and experiences
Gracefully accept constructive criticism
Focus on what is best for the community
Show courtesy and respect towards other community members
If you believe someone is violating the Code of Conduct, we ask that
you report it to The Carpentries Code of Conduct Committee by completing
this form.
Today’s Trainers
To begin class, each Trainer should give a brief introduction of
themselves.
Think of an example of a great a lesson that you have followed (were
taught in a class, read through online, read in a book). What did you
find was so good about it? Why did it make such an impression on you?
Try to differentiate between what was good about the
performance of the teacher/trainer and what was good
about the content of the lesson itself. Take a few minutes to
write down some notes about your answer, then introduce yourself to the
other participants and tell them about it.
Collaborative Lesson Development Training Overview
The main objective of this training is to teach you the skills you
need to design and develop an effective lesson, in collaboration with
other members of the community.
Collaborative Lesson Development Training is
taught over three days (six half days), with an extended break between
the first two days (“part 1”) and the last day (“part 2”). The training
in part 1 focusses on good practices in lesson design and development,
and the concepts and skills needed to build a lesson website with The
Carpentries lesson infrastructure. Part 2 of the training begins with a
reflective discussion of lesson deisgn and iterative development, before
shifting focus to the skills required for effective collaboration on
open source projects.
During the training, we will introduce the steps you can take to
design and develop a lesson to meet the needs of your target audience,
and give you time to begin implementing those steps during the workshop
itself. By the end of the training, you can expect to have defined an
outline for your whole lesson, and begun filling in a detailed plan for
some of its individual sections. The content you create in the training
will exist as an open source lesson website similar to the
one this training is based on. You should also know how you can
continue building on the lesson and some plans for how you will
collaborate on the project after the training ends.
We will focus on three main areas:
Designing a Lesson
Much of the training will discuss a process to incorporate good
practices in lesson design. We will explore how defining the specific
skills you wish to teach early on in the development process provides a
foundation from which you can build a stronger, more impactful
lesson.
Building a Lesson Website
Throughout the training, while you design and begin developing the
content of your lessons, we will teach you how to incorporate this into
an organised and accessible website using our lesson infrastructure.
Collaborating Effectively
We believe that lessons are much more likely to succeed, and to
remain useful in the long term, if they are developed collaboratively.
This training will discuss some of the ways that you can welcome new
collaborators and work effectively and efficiently with those you
already have.
Learning How to Teach a Lesson
This training will focus on the content - how to prepare a good
lesson. More about the performance - how to deliver a lesson most
effectively - is covered in The
Carpentries Instructor Training.
Key Points
This training aims to teach you a process for designing a lesson and
the skills to develop it as an open source website, in collaboration
with others.
What are the recommended steps to take when developing a new
lesson?
What lesson do you want to develop during and after this
workshop?
Objectives
After completing this episode, participants should be able to…
Explain the lesson design process we will be adopting for this
course.
Summarise the lessons that participants will be working on.
A Lesson Design Process
In order to design an effective lesson, we need a structured approach
with the learner in mind and clearly identified goals. Throughout this
training we will use a modified version of a process for curriculum
design commonly referred to as backward design, described by
Gill Nicholls1. Following Nicholls’ paradigm, we begin by
defining exactly what you want your learners to be able to do after
they have completed the lesson/training/course, with the subsequent
stages involving the design and evaluation of content that will directly
help learners meet those stated outcomes. By defining at the beginning
of the process what you want the outcomes to be at the end, you ensure
that your efforts remain focused on those goals as you work.
This training promotes an iterative backward design process where,
after identifying the target audience of our lesson, we
Define desired learning outcomes.
Design assessments to determine progress towards desired
outcomes.
Write content to lead learners from one of these assessments to the
next.
Assess learner progress towards outcomes during teaching.
(After the break) evaluate how closely the outcomes meet the
objectives.
An overview of the iterative process of lesson
design and development, adapted from Nicholls’ five phases, that will be
presented in this training.
The process throughout this training
Note the cyclical nature of the process described above. You will
complete one iteration through this cycle across the two parts of this
training (though probably for a limited part of your lesson, rather than
the whole thing):
Define desired learning outcomes. (Training part 1)
You will define learning objectives for your lesson as a whole and
for individual sections.
Design assessments to determine progress towards desired outcomes.
(Part 1)
You will learn about different types of assessment and how they can
give you information about your learners’ progress towards the defined
objectives.
You will design and implement exercises that are appropriate to your
target audience and the skills you want to teach them.
Write content to lead learners from one of these assessments to the
next. (Part 1)
You will choose examples and a narrative that can help learners gain
insight into the topic of your lesson.
You will begin to write content that is accessible, relevant, and
appropriate for your target audience.
Assess learner progress towards outcomes during teaching. (During
the break between parts)
You will deliver part of your new lesson and gather information
about how effectively it teaches learners what they need to know.
Evaluate how closely the outcomes meet the objectives. (Part 2)
You will revisit your lesson design and content and make plans to
update it, based on your own reflections and the feedback you gathered
from learners.
Teaching the new lesson content is an essential intermediate step in
the process, and the reason this training is delivered in two parts. The
importance of gathering feedback and reflecting on teaching experience
will be a common refrain throughout this training.
Your Lessons
This training will provide many opportunities for discussion of your
lessons. Providing some context now for the lessons that you will be
creating will help the Trainers and other participants get involved in
those discussions and give you feedback as you follow the process.
Discussion (10 minutes)
Share your answers to the following questions in the shared notes,
then discuss them with the Trainers, your collaborators, and the other
participants.
What is the topic of the lesson that you plan to develop based on
this training?
Have you created training material on this topic before?
What is motivating you to create this lesson?
Trainees should have received the questions above in advance and
arrived at the training with notes prepared. Where multiple trainees are
working on the same lesson project, ask for a volunteer or call on
someone to represent their team, leaving space afterwards for their
team-mates to speak up if their answers differ.
The main objectives of this discussion are:
to ensure that the members of each collaborative team share a common
vision for the lesson they want to create
to help Trainers and collaborators understand each trainee’s
motivation for attending the training and building their lesson
Iterative Development
The Carpentries community develops open source lessons, which can
always be updated and may never be finished. A lesson can
undergo many iterations before it reaches a relatively stable state. To
reflect this, we encourage lesson developers to indicate the status of
their lesson by labelling its progress through a lesson life
cycle:
The life cycle of a lesson
Each life cycle stage indicates the level of maturity of a
lesson:
pre-alpha: a first draft of the lesson is still
being constructed.
alpha: the lesson has been/is being taught by the
original authors, but has not been fully tested.
beta: the lesson is ready to be taught by
instructors who have not been significantly involved in its developed to
this point.
stable: the lesson has been extensively tested by
the authors and others. It can be considered broadly complete and
unlikely to undergo any drastic changes without warning.
Although your lessons will probably remain in pre-alpha throughout
this training, some of the content will be equally valuable at later
stages and we will also point you towards resources to help with testing
the lesson and gathering feedback.
Why is it so important to think about the target audience early in
the process?
How can you ensure that your lesson reaches the right audience?
Objectives
After completing this episode, participants should be able to…
Describe the importance of aligning lesson design with the intended
audience.
Compose a list of prior knowledge required to follow a lesson.
Target Audience
Given the limited time in a short-format training, it is vital to
define the scope of the lesson, i.e. what people need to know before and
what they will know after the lesson. Thinking carefully about the
target audience will help you with this and defining desired learning
outcomes (the first step of the lesson design). Prominently displaying a
description of the target audience will also help attract people with
the right motivation and relevant prior knowledge to attend your
workshops.
Expertise
One of the most important things we can identify about our target
audience is the level of expertise they will already have in relation to
the skills taught by your lesson. In The Carpentries Instructor Training
curriculum, we describe three
different stages of skill acquisition: novice, competent practitioner,
and expert and how it directly correlates to the complexity of mental
models these different groups have about a domain/topic.
Briefly, the novice is someone who does not know what they do not
know, i.e., they do not yet know what the key ideas in the domain are or
how they relate, the competent practitioner has enough understanding of
the domain/topic for everyday purposes, and the expert is someone who
can easily handle situations that are out of the ordinary and can
immediately use their prior knowledge or skills when presented with a
new problem in the domain.
When designing a new lesson, it is important to think about the level
of expertise that you expect learners to arrive with for two
reasons:
It helps to predict what prior knowledge and mental
model learners will have of the lesson domain when they arrive. This
can enable you to make progress quickly by
working to help learners recall (activate) that prior
knowledge1,
building on the conceptual understanding they already have2 and,
perhaps most importantly, giving you some idea of what
misconceptions they might arrive with. It is vital that
misconceptions are identified and corrected early on, before learners
try to incorporate new knowledge into a broken mental model. (More on
this in Stay On
Target.)
People at different stages of this process need to be taught
differently. For example, novices will learn more from lessons that
include worked examples and are more tutorial-like i.e. focused on a
specific task, with step-by-step explanations of the process, but
without a lot of extra information that is not directly relevant.
However, the same approach may actually hinder learning for competent
practitioners who may be distracted by a step-by-step explanation of
something they already have the prior knowledge of3. For learners at this
level of expertise, lessons which include activities offering learners
the freedom to explore options and develop their own solutions, are
likely to be more effective.
Motivation
Furthermore, your lesson will be more effective if it aligns with the
motivations of the target audience. Understanding the wants and needs of
your target audience, what they know already and what kinds of problems
they want to solve, will help you design a lesson that learners can see
the value in. It will give them the impression that taking the lesson
will be worthwhile (called positive expectancies in the
literature).
We will look more at strategies to establish value and build positive
expectancies in the next episode.
Be Specific
It can be tempting to identify a target audience only in vague terms,
for example by writing that a lesson is aimed at “PhD students” or
“early career researchers”. However, taking the time to focus on real
people, or imagined personae, who represent your target audience will
help you take time to consider the various aspects that can influence
how much someone will learn from your lesson. It will also help you
notice when the assumptions you are making about your target audience
are unreasonable.
Most of all, it will help you stay connected to the fact that you
are not your learners: they will arrive at the lesson with
different priorities, interests, and challenges than your own.
Lesson Design Notes Document
At this point, the Trainers will share the
template for a Lesson Design Notes document with trainees. Trainees
should make a copy of this document and fill in their lesson title. You
will populate other parts of the document with the notes and information
they produce throughout the training.
Exercise: thinking about target audience (15 minutes total for both parts)
Part 1 (all, 5 minutes): think about a member of the target audience
for your lesson, and answer the following questions in the context of
your lesson topic:
What is their background?
What do they already know how to do?
What do they want to do with the skills they will learn from your
lesson?
What problem will your lesson help them solve?
Share your answers with your collaborators. How do they compare? If
you have identified different audiences, are they compatible? Or would
your time be better spent focussing on one particular audience for this
lesson?
Take notes on your discussion in your Lesson Design Notes document.
It can be particularly helpful to note down any decisions made
e.g. potential target audiences that were explicitly discounted, and
your reasons for doing so.
Take notes about your choice of target audience in your Lesson Design
Notes document. It can be particularly helpful to note down any
decisions made e.g. potential target audiences that were explicitly
discounted, and your reasons for doing so.
Then write 1-2 diagnostic questions, for use before the lesson is
taught, to help you assess whether a respondent falls within the
intended audience for your lesson.
There is more to consider about your target audience than we could
capture with only the questions listed above. In your own time, you
should think more about the other considerations you might need to make
when writing a lesson for your audience.
For example, what vocabulary do they use? The terms you are teaching
in your lesson might have a different meaning in your learners’ domain
of expertise, and it can be helpful to prepare for and try to avoid
confusion arising from this clash. Furthermore, might their primary
language differ from yours? If so, how might this change the way you
write the lesson?
Defining Prerequisite Knowledge
A very common challenge encountered in workshops is heterogeneity of
expertise among the audience. When learners arrive at a workshop with a
wide range of previous experience with the topic, it is difficult for
the instructors to keep everyone engaged. Those who arrive with too
little relevant knowledge and experience can struggle to follow the
lesson content at the pace you expect, while those who arrive with too
much are likely to become bored and despondent as their expectation of
learning new skills is not met.
One way to try to guard against this is to publish the description of
your target audience when you advertise a workshop teaching your lesson,
alongside a list summarising the skills and conceptual knowledge you
expect learners to arrive with. Another is to use the information you
have about your target audience to ask questions of potential learners
when they apply/register to join the workshop (like the diagnostic
questionnaire you may have prepared in the exercise above), and use the
answers they give to filter out those who fall outside your intended
audience.
While valuable, this kind of pre-assessment should be approached with
caution: people are often bad at self-assessment i.e. estimating our own
ability to perform a task4. We can try to mitigate for this when
designing the questions for a pre-workshop survey, leaving little room
for inaccurate self-assessment to confound the results. But experience
suggests it is very difficult to ensure that every learner in a workshop
falls within the intended audience of a lesson.
Write a list of the skills/knowledge your learners will be required
to have before they can follow your lesson.
If you are struggling with this exercise because your lesson audience
is novices, think about skills like touch typing, using a web browser,
or interacting with a command line or graphical interface. These are
skills commonly overlooked by experts and competent practitioners.
Key Points
We recommend an iterative lesson design process that begins with
identifying the target audience, before defining learning outcomes, then
creating assessments, writing explanatory content, and evaluating the
lesson in a workshop.
Thinking about the target audience early in the design process helps
to ensure that your lesson is built around the needs and motivations of
real people.
Use the description of your target audience to help attract people
with the appropriate interests and prior knowledge to your lesson.
See chapter 1, How Does Students’ Prior Knowledge
Affect Their Learning?, of Ambrose
et al. 2010.↩︎
See chapter 2, How Does the Way Students Organize
Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
How can describing the things you intend to teach aid the process of
writing a lesson?
How can you be specific and realistic about what you will teach in
your lesson?
What are some of the risks associated with unrealistic or undefined
expectations of a lesson?
Objectives
After completing this episode, participants should be able to…
Explain the importance of defining specific, measurable, attainable,
relevant and time-bound objectives for a lesson.
Evaluate a written lesson objective according to these
criteria.
In this episode we will begin the first step of
our iterative design process: defining the skills and knowledge we want
learners to leave with.
At this stage of the training, you should have a clear idea of who
the target audience is for your lesson, and what knowledge, skills, and
abilities you expect them to arrive with. Now it is time to consider the
additional knowledge, skills, and abilities they will have by the time
they leave: these are the learning outcomes of your lesson. It
can feel strange to jump from one end of the process to the other like
this, but clearly defining your goals early in the lesson development
process is vital. As we will see in this episode, it helps you to
determine the activities, examples, etc. that are appropriate for the
lesson content, and provides a scope for what should and should not be
included.
Why Focus on Skills?
To ensure your audience stays motivated, and your lesson feels
relevant to them, we recommend that lessons focus on teaching skills
rather than tools. Lessons should be centred around what you are
empowering learners to do, what will be most beneficial to
them, rather than a list of functions or commands you are teaching them
to use. Placing the emphasis on skills over tools will help you
prioritise key concepts and consider how your lesson can have the
biggest impact on the way learners do their work.
Learning Objectives
The desired outcomes (the learning objectives) of a lesson
should be new skills, i.e. things that the learner can do. For the vast
majority of lessons, these will be cognitive skills: things
learners can do with their minds. (Lessons intended to teach other kinds
of skill, such as woodwork, playing a musical instrument, or making
sushi, are probably better suited to a different platform than a static
website.) Cognitive skills cannot be equally easily acquired: before we
can apply concepts and create something new, we must
attain the ability to remember and distinguish between
new concepts. Remembering and distinguishing are also abilities that are
often faster to gain than applying or creating.
We must try to be realistic about how far along this scale we can
move learners during a single workshop/lesson. This is one reason why
the target audience is so important: if we can predict what learners
will know when they arrive at the lesson, we can better define the
outcomes we can expect when they leave.
Defining objectives for a lesson is essential because it allows us to
focus the rest of our time on developing content that is necessary for
learners to reach these goals. It will help us ensure we do not miss
anything important or, conversely, include anything superfluous that
could use up valuable time or distract instructors and learners.
What Does an Objective Look Like?
This diagram highlights the most important
elements of a learning objective.
Objectives can be defined for a lesson as a whole - what should
learners be able to do at the end of a workshop teaching this lesson? -
and for individual sections within it - what should learners be able to
do after following this particular part of the lesson?
This is probably a episode-level objective because it is very
specific. A corresponding lesson-level objective might be “read data and
libaries into the python environment”. It is still fairly specific but a
bit broader in scope and contains several episode-level objectives.
The lesson level objective for the current section
of this training is: define SMART learning objectives.
This objective also covers a later episode as well.
The episode level objectives for the current section
of this training are:
Objectives for This Section
Explain the importance of defining specific, measurable, attainable,
relevant and time-bound objectives for a lesson.
Evaluate a written lesson objective according to these
criteria.
These should be read as if they were endings to a sentence
beginning
“At the end of this session, learners should be able to …”
Each objective starts with a verb and describes one (and only one)
skill the learner will obtain.
For objectives to be as helpful as possible, they need to be written
in a way that will allow us to directly observe whether or not a learner
has attained the skills we want them to. This means that the skills
described by our objectives should be measurable: as a general rule,
action verbs such as “explain,” “choose,” or “predict,” are more helpful
than passive verbs such as “know,” “understand,” or “appreciate”, which
are hard to directly assess and are often open to interpretation.
A popular aid for defining learning objectives is Bloom’s
Taxonomy, which divides cognitive skills into several categories.
The original taxonomy arranged these categories in a strict hierarchy,
which has since been disputed. Regardless of whether these skills
conform to such a hierarchy, Bloom’s Taxonomy serves as a very useful
bank of action verbs for use in learning objectives.
Bloom’s taxonomy - a framework for categorising
educational goals, image
from Wikimedia Commons reused under CC BY 4.0 license
In the context of a lesson, SMART objectives should be:
Specific: they should clearly describe a particular
skill or ability the learner should have.
Measurable: it should be possible to observe and
ascertain when the learner has learned the skill/abilities described in
the objectives.
Attainable: the learner should realistically be
able to acquire the skills or abilities in the time available in a
workshop/by following the text of the lesson.
Relevant: they should be relevant to the overall
topic or domain of the lesson as a whole.
Time-bound: they should include some timeframe
within which the goal will be reached. For learning objectives, this is
built into the approach described above.
Note that, for any lesson/curriculum that will be taught in a fixed
amount of time, attainable and time-bound are
overlapping: learning objectives for your lesson will answer the
question “What will learners be able to do by the end of this lesson?”
and the time available to teach the lesson, combined with the expected
prior knowledge of your target audience, will determine how attainable
they are.
You may find this template of a Markdown table useful to share with
your trainees before they complete the next exercise.
Look at the example learning objectives below. Fill in the table for
each objective, checking off the cells if you think an objective meets
the criteria or leaving it unchecked if not. You should assume each
objective is for a lesson to be taught in a two-day workshop. Note down
any observations you make as you move through the list. If you have
time, try to imagine the titles of lessons that would have these
objectives. This part of the exercise should take 10 minutes.
At the end of this lesson, learners will be able to:
create formatted page content with Markdown.
program with Rust.
fully understand GitHub Actions.
Objective
Action verb?
Specific
Measurable
Attainable
1
2
3
In the last five minutes of the exercise, we will discuss as a group
how each objective might be improved.
Objective
Action verb?
Specific
Measurable
Attainable
1
✅
❓
❓
✅
2
❓
❌
❌
❓
3
❌
❌
❌
❓
Objective 1 is the closest to what we ideally want in a lesson
objective, but it illustrates how difficult it can be to make an
objective truly specific. For example, a more specific and measurable
version of this objective could be:
write links, headings and bold and italicised text with
Markdown.
Lesson Scope
One of the major challenges of lesson design is choosing what to
include in a lesson: what the main points will be, in what order they
will be introduced, how much detail can be provided, and how much time
can be spent on each point. Especially when writing lessons for short
form training like a Carpentries workshop, difficult decisions often
need to be made about what can and cannot be included. Trying to fit too
much content into a lesson is counter-productive2, so it is good to
avoid the temptation to cram in more content than you have time to cover
properly.
Writing learning objectives is a good opportunity to begin thinking
about this lesson scope, and can provide assistance when you are faced
with a difficult decision about what content to cut out.
For instance, consider the order in which new skills must be
acquired. Before learners can begin to acquire “higher-level” cognitive
skills to perform creative and analytical tasks, they must first acquire
the foundational knowledge and conceptual understanding of the domain.
Furthermore, these higher-level skills take longer to acquire so, unless
you can expect your target audience to arrive at the lesson with the
relevant foundational knowledge and understanding, it is probably
unrealistic to aim to have learners completing creative tasks before its
end.
As should become clear through activities in the upcoming episodes,
lessons can be broadly considered as blocks of content associated with a
particular learning objective. This can be helpful when making choices
about content to remove, because the task can be considered in the
context of taking out whole learning objectives.
Defining Learning Objectives
We have discussed the importance of defining objectives early in the
lesson design process, and looked at some examples of objectives written
for other lessons. Now it is time to begin defining objectives for your
own.
Here are some recommendations to help you get started:
Aim for 3-4 lesson objectives for every 6 hours your lesson will
take to teach. (For example, this curriculum is designed to be taught in
~18 hours and has ten learning
objectives.)
These objectives are for your lesson as a whole: try to define the
“end point” knowledge and skills you want learners to acquire. You can
think of these as the things you would test in a fictional “final exam”
your learners would be able to tackle.
Later you will write “episode-level” objectives that should define
intermediate steps towards the high-level objectives you identify for
your lesson here.
Exercise: defining objectives for your lesson (20 minutes)
Write learning objectives for your lesson into the relevant section
of your Lesson Design Notes document - what do you want learners to be
able to do at the end of the workshop? When writing these lesson-level
objectives, try to follow the SMART framework: make them specific,
measurable, attainable, relevant, and time-bound.
Take notes about the your discussion in your Lesson Design Notes
document. A record of the decisions you made and your reasons for
choosing these objectives can be very helpful for you and others to
understand the design and scope of the lesson later.
If you find yourself writing very specific episode level
objectives, keep them in your notes and try to think of a slightly
broader objective that may contain several very specific objectives. We
will come back and work more on episode objectives next in the
training.
Swap objectives written in the previous exercise with a partner (you
can also explain or show them what you wrote about your target audience,
but this not essential) and review them with the following questions in
mind:
Are the objectives clear?
Do they use “action” verbs?
Could you directly observe whether a learner had reached this
objective?
Where do the skills described in these objectives sit on the
scale?
(optional) Are these objectives realistic, given the target audience
of the lesson?
Advertising your lesson
These learning objectives, as well as the list of
prerequisite knowledge you defined earlier, are very useful
information to include when advertising a workshop that will teach your
lesson. It will help people understand whether or not the event is a
good fit for them, and manage their expectations about what they will
learn if they attend.
Key Points
Defining objectives for a lesson can help to focus your content on
the most important outcomes, and outline the scope of the project.
Following the SMART framework can help make your learning objectives
as useful as possible.
Leaving objectives unrealistic or undefined increases the risk of a
lesson losing focus or spending time on activities that do not help
learners gain the most important skills.
What considerations are there when choosing an example dataset for a
lesson?
Where can I find openly-licensed, published data to use in a
lesson?
Objectives
After completing this episode, participants should be able to…
Find candidate datasets to use in a lesson.
Evaluate the suitability of a dataset to be used in a lesson.
Choose examples that will prepare learners for formative assessments
in the lesson.
Develop a story
With your high-level lesson objectives set, now is a good time to
consider any additional resources you may need to effectively
communicate your message to learners before you dive deeper into writing
the lesson content.
Creating a Narrative
Writing your lesson as a story helps learners stay motivated and
engaged. The story you create can also help learners more easily connect
how the skills they are learning now could be useful after the workshop.
You can enable learners to make connections between what they learn in
your lesson and their own work by creating a narrative that resembles a
situation the learners might encounter there.
Here are some example narratives from lessons:
Software
Carpentry Git lesson uses the story of Alfredo, a chef working with
his team to create a repository of his favorite recipes.
Building
Websites With Jekyll and GitHub lesson uses the narrative of
creating a website for a research project to teach authoring webpages
with Markdown and compiling them into a website served on GitHub.
Sometimes, a single and continuous narrative threads through the
entire lesson, while other times, you use a mini-narrative to teach a
specific point in an episode. For example, within the overall narrative
of creating a research project website in Building
Websites With Jekyll and GitHub lesson, there is a mini-narrative of
creating a series of short blog posts to teach how to extract a class of
structurally similar pages into a common layout template and filter out
common structural components of pages.
You may find that images and figures enhance your narrative. Images
are powerful communicators, conveying a lot of information with few or
no words. Images can also be distracting for learners. If you choose to
include images as part of the narrative of your lesson, be sure that
they are consistent with the situation you are describing so that they
do not increase learners’ cognitive load.
Depending on the tool you are teaching, you might also include a
particular dataset as a part of the story you are weaving into your
lesson. It is common for lessons to include a dataset that is used in
examples and discussed throughout. This can help you maintain a
narrative flow and make the lesson feel more authentic. Even if your
topic doesn’t require a dataset, deciding on a consistent narrative will
help create a flow between episodes and reduce cognitive load for
learners.
Use varied examples if you want to teach abstract
concepts.
The focus of this episode is on finding one central example to be
used throughout a lesson. However, where you wish for learners to
develop understanding of more abstract concepts, you should try to
provide a variety of examples with a common theme. This could be
achieved by providing multiple opportunities to recognise and apply the
concept in taught examples, activities, and exercises.
Finding Images
Copying an image from a website is technologically simple but can be
legally and ethically complex. Images are intellectual property and are
subject to intellectual property laws including, but not limited to,
copyright and trademark laws. These laws differ by country but are
consistent in theme: do not take intellectual property that does not
belong to you without permission.
When looking for images that illustrate the narrative of your lesson,
avoid copying images that do not include a reuse license. Assume that
you cannot reuse these images unless you seek written permission from
the image creator or owner. Instead, look for images that indicate that
they are in the public domain or carry a permissive reuse license such
as CC0 or CC-BY. Public domain images can be freely reused and adapted.
Images carrying a reuse license can be used and adapted in accordance
with their license terms.
If you cannot find reusable images that match your narrative, you can
create your own images or seek help from others in the Carpentries
community. When incorporating original images into your lesson, be sure
to license these images to be compatible with the license on the rest of
your lesson materials.
The guidance in this section is not a substitute for legal
advice.
Compatible Licenses
Creative Commons offers a chart that identifies which CC licenses are
compatible with each other for adaptation (“remix”) purposes: Creative
Commons license comparison chart.
GNU offers commentary about a variety of licenses for free software;
this resource may be valuable when considering a license for code: GNU: Various
Licenses and Comments about Them.
Finding a Dataset
Testimonial
It was very distracting to me in a workshop, when we were given a
dataset to work with but no-one explained what that data was and what it
meant. It prevented me from following what the instructors were actually
trying to teach me, while I tried to figure out the meaning of the
numbers, columns names etc on my own.
When searching for a dataset to use in your lesson, there are a
number of factors to consider. First, if possible you want the dataset
to be an authentic use case for researchers in your target audience.
This means balancing the size and complexity of a dataset while avoiding
additional cognitive load for learners. The dataset may need a certain
number of observations to demonstrate some of the skills you are
teaching or have a number of variables that is sufficiently similar to
what learners may encounter in their work to feel authentic. At the same
time, you want your learners to be able to interpret the data fairly
easily during the lesson so they can focus on the skills you are
teaching. This may mean you will need to subset your dataset by removing
some observations (rows) or variables observed (columns).
You may also want to include and review a data dictionary in your
lesson, explicitly taking the time to review the information included in
the dataset. For inspiration, see the Social
Sciences Data Carpentry data dictionary. An additional factor to
consider when choosing a dataset to include is the license. You want to
find a dataset where the data provider allows for you to freely use it.
The best option is a dataset with a CC0 (Public Domain Dedication)
license, as other licenses may have more ambiguity around data reuse.
Lessons included in The
Carpentries Incubator are encouraged to use CC0 licensed data, and
may be required to do so to qualify for peer review in The Carpentries Lab. Even with a
CC0 license, you will still want to follow best practice in giving
attribution to the data provider or collecting agency.
The Ecology Data
Carpentry lessons’ dataset comes from the Portal
Project Teaching Database. This dataset is an actual ecological
research project’s data that was simplified for teaching. The reuse of
this dataset throughout the Data Carpentry Ecology lessons helps stitch
together the process of data analysis throughout the workshop, from data
entry and cleaning to analysis and visualization.
The Social
Sciences Data Carpentry lessons’ dataset is the teaching
version of the full Studying African Farmer-Led Irrigation (SAFI)
dataset. The SAFI dataset represents interviews of farmers in Mozambique
and Tanzania, conducted between November 2016 and June 2017. The
interviews surveyed household features (e.g. construction materials used
for dwellings, number of household members), agricultural practices
(e.g. water usage) and assets (e.g. number and types of livestock). The
teaching version of the SAFI dataset has been simplified and
intentionally “messed up”” to enable demonstrating common data cleaning
issues often found in real-life data.
Patient
inflammation dataset - from the Software Carpentry Python novice
and the incubating
intermediate lessons - is used to study the effect of a new
treatment for arthritis by analysing the inflammation levels in patients
who have been given this treatment.
Data Carpentry’s Astronomical Data
Science with Python lesson uses two astronomical datasets, from the
Gaia satellite and the Pan-STARRS photometric survey, to reproduce part
of an analysis described in a published article.
When deciding on a dataset, it is also important to consider the
ethical use of each dataset. Does the data contain personally
identifiable information?Was the data collected without
permission from the groups or individuals included?Will the
data be upsetting to learners in the workshop? If the answer to any
of these questions might be yes, you will need to do more research on it
and continue to look for other options. Commonly data is misused from
historically excluded and exploited groups. The CARE Principles for
Indigenous Data Governance1 (Collective Benefit, Authority to Control,
Responsibility, and Ethics) are good starting point for thinking about
data sovereignty and considering the ethics of data collected about an
individual or groups of people.
Outline covering the above paragraph - also used in CodiMD
template.
Dataset Considerations
Ethical use (see prompts below)
License - CC0 Recommended
Complexity - Is it easy to understand? Is it sufficiently
authentic?
Number and types of variables
Questions about Ethical Use of Datasets
Does the data contain personally identifiable information?
Was the data collected without permission from the groups or
individuals included?
Will the data be upsetting to learners in the workshop?
Examples of Public Repositories
When looking for data to reuse, consider public repositories in the
subject area for your lesson or general data repositories such as:
Exercise: Choosing a Dataset or Narrative (30 minutes)
Referring to the advice given above, find an appropriate dataset or a
narrative for your lesson. Identify one or more potential candidates and
note down the advantages and disadvantages of each one. As a reminder,
here are some aspects we suggest that you consider:
For datasets:
size
complexity
“messiness”/noise
relevance to target audience
availability
license
ethics
For narratives:
authenticity
relevance to target audience
complexity
possibility to teach useful things first/early
Takes notes in your Lesson Design Notes document about your
discussion and the decisions made. It may be particularly helpful to
record:
Which datasets and narratives did you consider?
How and why did you choose between them?
What implications do you think your choice of dataset and/or
narrative will have for the design and further implementation of your
lesson?
Where to Store your Example Data
GitHub is not a good place to store data, especially when it is large
and/or does not consist of text files. Instead, we recommend that you
publish your example data elsewhere and link to it from your lesson
website. This has the added advantages that you can publish the data
under its own license (ideally CC0, as discussed above), obtain a
separate DOI for it, and create another backup of your data. Dryad, Figshare, the Open Science Framework, and Zenodo are good general platforms for
publishing data. However, if your lesson covers a particular domain with
its own established standard for publishing data, we recommend that you
use that. The Generalist
Repositories Ecosystem Initiative (GREI) includes several more
general options, and provides a decision tree to help you
choose the most appropriate location for your data.
When you publish the data for your lesson, make sure to include:
a description of each of the files included.
information about their provenance.
the lesson they are used in.
the license terms.
anything else you think people need to know about the data.
People learn faster if they are motivated2, and learners will be
motivated if you teach most useful things first. As you
think about what story your lesson will tell, it is important to put the
pieces that are most interesting to learners up front. If they are able
to quickly learn tools or skills that they see as useful from your
lesson, they will be more interested in continuing to learn other
concepts that are needed. You may notice this trend in many of the
Software Carpentry lessons. In particular, many of the coding language
lessons (R and Python) have the learners create a plot very early in the
lesson and then go back and teach coding fundamentals such as loops and
conditionals. Visualization of data is often a very motivating and much
needed skill by learners. Getting to visualization early, keeps learners
interested in learning the additional and vital skills where the
application might be less clear.
Remember, even if you do not need a dataset for your lesson, you
should decide on a narrative. Centering your lesson around a central
example reduces the cognitive load of switching between examples
throughout the lesson. Using an authentic, yet simple, dataset will also
help reduce cognitive load and help learners to see how they might apply
what they learned to their own projects. It is also important to
consider licensing and ethical considerations when looking for a lesson
dataset.
Key Points
Using a narrative throughout a lesson helps reduce learner cognitive
load
Choosing a lesson includes considering data license and ethical
considerations.
Openly-licensed datasets can be found in subject area repositories
or general data repositories.
How can the objectives for a lesson be used to break its content
into sections?
How should objectives be written for a smaller part of a whole
lesson?
Objectives
After completing this episode, participants should be able to…
identify appropriate parts of their lesson to break into individual
sections
define learning objectives for a section of a lesson.
So far, you have defined the learning objectives for your lesson as a
whole, and came up with a narrative or example datasets that will help
you tell your story.
Rather than write the lesson as a single, long document, we recommend
that you break it up into chunks like chapters in a book or episodes in
a season of a TV series. This can help manage learners’ cognitive load
by ensuring that you organise content into coherent, more self-contained
chunks, and makes it easier for instructors to schedule regular and
frequent breaks while teaching. Thinking about how content can be broken
down like this early in the lesson design process helps you to consider
the path learners will take to reach your defined objectives, and
identify the component skills and knowledge they will need to pick up on
the way.
In The Carpentries, we refer to these individual parts of a lesson as
episodes, to encourage lesson developers to think about them as
self-contained units (with their own narrative arc) that nevertheless
contribute to the larger whole (the theme or story that runs through a
full season). It also helps to think about the typical length of an
episode: these chunks contain 20-60 minutes of content (teaching +
exercises).
Each episode will exist as a page in the website we will build for
our lessons.
Planning Your Episodes
Before we can start creating the episodes of a lesson, we need to
spend some time planning out the number and order of them. The learning
objectives you defined for your lesson can help with this: at the very
least, it is probably a good idea to have one episode dedicated to each
objective. You might also find that you can “decompose” the lesson-level
objectives into more finely-grained steps that learners can take towards
those end points. For example, the lesson-level objective
“create formatted page content with Markdown”
may be broken down into,
“create bold, italic, and linked text with Markdown”
“explain the different header levels in Markdown”
“add images with a caption and alternative text description to a
Markdown document”
Some questions you might ask yourself to help break down your
lesson-level learning objectives include:
What new knowledge and skills will learners need to acquire to be
ready to fulfil the overall objectives for your lesson?
What order should these concepts and skills be introduced in? Are
some dependent on others?
If some of these concepts and skills are complex, can they be broken
down even further?
Exercise: Defining Episodes for a Lesson (25 minutes)
With your team, in the shared notes document for your lesson:
Based on the lesson-level objectives and your knowledge of the
lesson topic, divide the lesson up into logical blocks (episodes), that
should each take approximately 20-60 minutes to teach. Think of these
logical blocks as topics that you need to cover within your lesson but
do not go too deep into defining learning objectives for individual
episodes - we will cover that soon.
Next, assign responsibility for one of these episodes to each
collaborator in your team. They will focus on this episode for the rest
of this training, and you will teach these episodes in a trial run
between parts 1 and 2 of this training.
Some questions you might ask yourself to help break down your
lesson-level learning objectives include:
What new knowledge and skills will learners need to acquire to be
ready to fulfil the overall objectives for your lesson?
What order should these concepts and skills be introduced in? Are
some dependent on others?
If some of these concepts and skills are complex, can they be broken
down even further?
If the length of the list of episodes you create is smaller than the
number of collaborators you have on your team, try assigning two people
to a single episode, with each taking responsibility for a subset of the
objectives defined for that episode. If you plan more episodes than you
have people in your team, do not assign more than one episode to each
collaborator for now, but we strongly recommend that you assign
yourselves consecutive episodes at the beginning of your
lesson.
Defining Episode-level Learning Objectives
Now that we have broken those lesson-level learning objectives into
episodes, we can use the same approach for the individual episodes of a
lesson, defining objectives for the episode to make clear what we intend
to teach in that section. Defining these objectives before writing
the episode content helps us to:
stay focused in the episode, without spending time on non-essential
topics
determine whether learners are attaining the skills we wish to teach
them (we will discuss this more in the next two episodes)
summarise the skills the learner can expect to gain by following
this section of the lesson
As before, here are some recommendations to help you define these
episode-level objectives:
Aim for 2-4 objectives per episode. If you need to write more than
that, consider breaking this section down into multiple episodes.
To ensure your episode content aligns with the overall goals for the
lesson, try to identify which lesson-level objective(s) your episode
will contribute to.
In the context of the previous recommendation, consider how the
objectives for your episode fit as a component of the higher-level
skills/understanding defined in the lesson-level objective(s).
Exercise: define objectives for your episode (30 minutes)
Using the same approach as you did for your whole-lesson objectives,
define a set of SMART objectives for your chosen episode. (15
minutes)
Compare your list with those created by your collaborators on the
lesson:
are there any gaps in these objectives, i.e. anything that should be
covered in these episodes but is not captured in the objectives?
are there any overlaps, i.e. anything that looks like it will be
covered more than once?
As a group, discuss how you will address any problems identified in
the previous step, and edit your objectives accordingly.
Optional Homework: Reflection Exercise (not included in timing)
Take some time to think back on what has been covered so far, then
make some notes on the most important points and actions you want to
take away from that. The Trainers and other participants will not look
at this - it is only for you.
If you do not know where to start, consider the following list for a
starting point:
draw a concept map, connecting the material
draw pictures or a comic depicting one of the concepts covered
write an outline of the topics we covered
write a paragraph or “journal” entry about your experience of the
training today
write down one thing that struck you the most
Key Points
Learning objectives for a lesson can help you split up its content
into chunks
How can you measure learners’ progress towards your lesson
objectives?
Why are exercises so important in a lesson?
What are some different types of exercises, and when should they be
used?
Why should we create assessments before we have written the
explanatory content of our lesson?
Objectives
After completing this episode, participants should be able to…
Describe the importance of regular exercises while a lesson is being
taught.
Choose the format for an exercise based on the outcome it is
intended to measure.
In this episode we move to the second of our
iterative design process: designing assessments to measure learners’
attainment of the objectives we defined previously.
As we have seen previously, defining objectives for a lesson (or a
teaching episode) can help to focus your content on the most important
learning outcomes and outline the scope of your lesson project. The goal
of the remaining steps of lesson development is to ensure that what
learners learn from following your lesson matches its defined objectives
as closely as possible. To do so, you need to develop assessments to
monitor progression towards your learning outcomes.
In order to measure progress and evaluate if and what learning
occurred, we have various types of assessment available to us:
summative assessments - used to verify whether
learners achieved the stated learning objectives after
instruction.
formative assessments - used to detect changes in
learner understanding or performance during instruction, to
provide feedback and insight (to both instructors and learners) into the
learners’ developing mental models of the topic taught and to identify
any old or developing misconceptions.
Summative assessments sum up what learning has been
achieved after training (e.g. via exams). They give valuable data about
learning attainment by individuals and entire cohorts but are not used
to guide further progress of a lesson. They may not be as suitable for
short courses, but may be necessary for those that give marks/grades or
certificates of completion.
Formative assessments are applied throughout a
course and with several different purposes: they provide a way to move
new
information from working memory to long-term memory; they can inform
instructors’ decisions about how to modify instruction to better promote
learning; they also inform learners about changes they may need to make
to improve their learning. Ideally, they should be used often
(e.g. after every 15-20 minutes of teaching), providing opportunities to
instructors to change pace and refocus learners’ attention. For short
courses, formative assessments are usually more valuable and easier to
implement in practice than summative assessments - they need not be
complex or time-consuming, just informative enough about learning for
both instructors and learners.
The most effective way to test learner understanding and progress is
to do such assessments in class - they engage all learners and allow
instructors to check learners’ confidence with the content and its
delivery, can help you deal with any potential misunderstandings as soon
as they arise, and maximise the value of workshop for everyone. Such
formative assessments also help with metacognition - the
awareness a learner has that they are succeeding in learning something
new.
Any instructional tool that generates feedback and is used in a
formative way to check for learners’ understanding can be described as
“formative assessment”. For example,
checking in - gauging learners’ satisfaction and
understanding using agreed signals (e.g. raising different coloured
post-it/sticky notes or Zoom
reactions to indicate that the pace is too fast/slow, that they
completed/have not completed an exercise).
group or collaborative discussions - raising
questions for discussion among the group. For example, think,
pair, share, where learners think individually about
an answer to a question, pair up with a classmate to discuss
their answer, and then share out the consensus they came to
with the class.
problem-solving or diagnostic exercises - setting
(coding or non-coding) challenges for learners to tackle, testing
comprehension of a subject and diagnosing any forming misconceptions
(e.g. via multiple choice questions).
individual (guided) reflections at the end of a
session to help process what was learned, for example asking learners to
write down, draw, or diagram (concept map) things they learned, noting
how things relate to one another or connect with previous knowledge,
things they want to know more about and any questions they still
have.
Think back through the parts of this training you have followed so
far. Identify two examples of formative assessment that the Trainers
have employed so far. As an extra challenge, try to decide whether these
assessments were used to assess progress towards a particular learning
objective and, if so, what the relevant objective might have been.
Some examples of formative assessments used so far in this
training:
this exercise aims to assess how well trainees are able to identify
the aspects of a target audience that influence the design of a lesson.
It also aims to expose any inconsistencies between the visions of the
target audience held by different collaborators.
Your trainers have probably checked in with the group at various
points in the training.
Although these check-ins are not specific to a particular objective,
they help give us an impression of how well trainees are able to follow
what we are teaching.
This helps us evaluate trainees’ progress towards the learning
objectives we have set in relation to the lesson infrastructure.
Exercise Learners’ Memory
In a simplified model of memory individuals are equipped with two
types of memory: working (also called short-term) and long-term.
Long-term memory essentially has unlimited storage but is slow to
access, whereas working memory is quicker to access but can only hold a
limited number of items at a time. For teaching, the goal is to help
learners move the new things they’ve learned from working memory into
long-term memory. Exercises help with this by providing learners an
opportunity to practice what was recently learned. Exercises should
occur frequently throughout the lesson to free up working memory and
make space for more new information.
Creating exercises builds upon the learning objectives you created
earlier in the lesson design process. You can design exercises based on
the actions/skills you described in your learning objectives (the
learning outcomes you intend for the lesson). This will be easier if
your wrote learning objectives with specific action verbs. Specific
verbs can help you decide what action you want the learners to perform
in the exercise. E.g. actions such as “explain” and “describe” may be
better assessed by discussions and multiple choice questions, while
“solve,” “construct,” “test” and other higher-level cognitive skills may
be better assessed by debugging tasks, code-and-run,
or use-in-a-different-context exercises.
Exercise: Exercise Types and When to Use Them (15 minutes)
The Trainers will assign you to pairs or small groups, and give each
group an exercise type to work on. Each group should assign a notetaker,
to summarise your discussion at the end of the exercise.
Then, considering the exercise type in general, as opposed to the
specific example given in the text, discuss the following questions
together:
What kind of skills would an exercise of this type assess? Try to
identify some action verbs like those we used to write lesson objectives
earlier in the workshop (you may refer to the extended Bloom’s
taxonomy for computing to help you).
Would this type of exercise be suited to a novice audience? Or is it
a better fit for intermediate or advanced learners?
Would this kind of exercise work well in an in-person workshop
setting? Would it be better suited to self-directed learning and/or a
virtual workshop?
Share the major points of your discussion in the collaborative notes
document.
The Trainers will assign your group a type of exercise to focus on.
Read about your given exercise type on the indicated pages of Is
This a Trick Question?:
multiple choice (page 13)
true-false (pages 20 & 21)
fill-in-the-blank (page 34)
authentic assessment (pages 46 & 47)
Then, discuss the following questions together:
Could exercises of this type be used in your lesson?
If so, can you identify any of your written objectives that could be
assessed with an exercise of this type?
Have your Trainers set you any exercises of this type in this
training so far?
Would this type of exercise be suited to a novice audience? Or is it
a better fit for intermediate or advanced learners?
Would this kind of exercise work well in an in-person workshop
setting? Would it be better suited to self-directed learning and/or a
virtual workshop?
Exercise: Exercise Types and When to Use Them
(15 minutes) (continued)
Trainers will lead a discussion about your findings for your chosen
exercise type with the rest of the participants.
Both of the resources linked from the exercise above, the
Exercise Types chapter of Teaching Tech Together
and Is
This a Trick Question? are worth reading in full. They collect
a lot of insightful discussion and illustrative examples together, which
can prove very useful when designing exercises for your lesson.
As you discussed with your group in the last exercise, different
types of learning objectives work better for novices, while others are a
better fit for competent practitioners or experts.
This can be understood in terms of the types of exercises that suit
the objective: exercise types that help manage cognitive load for the
learner, such as fill-in-the-blanks,
faded
examples or Parsons
problems (which all provide a lot of the guiding process/scaffolding
code and allow the learner to focus on a specific concept or skill) are
a good fit for a novice, to whom all elements of the topic are new.
However, these kinds of exercise do not provide an opportunity for
learners to develop higher-level skills, such as the ability to create
whole new functions or scripts, or to extrapolate from the examples they
have seen to solve a different kind of problem. Indeed, example and
exercise types that are helpful to novices may even be
counter-productive for learners with a greater level of expertise1.
Thus you want to choose your objectives to fit your intended audience
and your exercise formats to fit your objectives.
An Expert’s View on Objectives and Audience Expertise
“Different types of lesson objectives (LOs) are better fit for
novices, while others are better fit for competent practitioners, etc.
and if exercises (formative assessments) are well aligned to LOs, [they]
will automatically serve the corresponding audience. Thinking in terms
of LOs (What should a learner do in order to achieve this specific LO?
Is this LO exactly what learners are expected to achieve by the end of
this piece of instruction? etc.) is easier than thinking in terms of LOs
+ audience + content. LOs should be tailored to the audience, and, if
this is well done, you may stop worrying about the audience. Create LOs
for the specific audience and create assessments for specific
LOs.”
- Dr. Allegra Via, Carpentries Instructor Trainer
Detecting and correcting misconceptions
and fixing learners’ incorrect/broken mental
models is as important as presenting your learners with new
knowledge and correct information. When mental models are broken,
learning can occur slower than you might expect2. The longer a prior
incorrect model is in use, and the more extensively it has to be
“unlearned”, the more it can actively interfere with the incorporation
of the new correct knowledge (since it will contradict the
misconceptions already present in the mental model).
We recommend that you read the sections of the Instructor Training
curriculum and Teaching Tech Together linked above. You may
also find it interesting and helpful to review our supplementary page
digging into misconceptions and
multiple choice questions in more detail.
Exercise: Assessing Progress Towards an Objective (30 minutes)
Using one of the exercise formats you have learned about so far,
design an exercise that will require learners to perform one of the
actions described in the objectives you wrote earlier, and that assesses
their ability to do so.
These should be assessments of the lower-level objectives defined for
individual episodes in the lesson, as opposed to the lesson-level
objectives you wrote first.
Trainees working as a team can choose whether to work together on
discussing and designing a single exercise to assess a single objective,
or to divide their efforts and each focus on an exercise for their own
episode. If you choose to take the latter approach and finish with time
to spare, spend the remainder reviewing and providing feedback on one
another’s assessments.
The same approach to designing exercises within a lesson can also be
used to create a short “pre-assessment” questionnaire for potential
learners to complete when they register for a workshop teaching the
lesson (or for self-evaluation before following the lesson on their
own). You can use the list of
prerequisite knowledge that you defined earlier to help with
this.
If you collect the results of this questionnaire, use it to follow up
with people who have registered for the workshop but do not fit the
intended target audience, to manage their expectations about how useful
the workshop will be for them.
You should aim to create all your assessments before you have written
the explanatory content of your lesson (recall Nicholls’ backward
design). These assessments will guide your lesson design process by
knowing exactly which knowledge you’d expect from your learners at any
point in the lesson.
Well-designed exercises are one of the most valuable resources for an
instructor and any time spent on this during lesson design is well
invested.
Key Points
Assessments are a way to determine whether the objectives you
defined for the lesson have been reached.
Exercises help learners commit what they’ve learned to long-term
memory.
Some types of exercises are better for particular audiences and to
address certain objectives.
Formative assessment happens during
teaching and provides feedback both to an instructor and a learner
- about progress and whether learning of new concepts occurred but also
about any misunderstandings and misconceptions which can hinder further
learning.
See chapter 1, How Does Students’ Prior Knowledge
Affect Their Learning?, of Ambrose
et al. 2010.↩︎
See chapter 1, How Does Students’ Prior Knowledge
Affect Their Learning?, of Ambrose
et al. 2010.↩︎
How can I prioritise what to keep and what to cut when a lesson is
too long?
Objectives
After completing this episode, participants should be able to…
Estimate the time required to teach a lesson.
Summarise the content of a lesson as a set of questions and key
points.
Connect the examples and exercises in a lesson with its learning
objectives.
Now that we have designed assessments to measure
attainment of the objectives set for the lesson, it is time to begin
developing teaching content to give learners the knowledge and skills
they need to succeed in those assessments.
Writing Explanatory Text
Explanatory text (including examples) help connect your exercises
together into a cohesive lesson. If your exercises are train stations
and scenic views, the explanatory text is the train tracks that connect
them. Often times the examples and exercises may seem like a good draft
for you to teach. However, the explanatory text makes it possible for
other instructors and self-directed learners to follow along by
providing all the relevant information directly in the lesson. It can
also help you remember your goals and stay on track teaching.
How much text to include is often a question of personal style while
balancing the needs of potential users, both other instructors and
learners. You want to find the balance between providing enough
information for learners to meet the learning objectives without
increasing cognitive load with extraneous information. In this section
we will discuss factors to consider when you are writing explanatory
text.
There are trade-offs to consider when preparing a site for use as an
instructional guide vs use as a self-directed learning resource.
Typically, self-directed learning resources are more verbose where
instructional guides are aimed at an audience that can fill in the gaps
on their own. However, very sparse text is less likely to be re-usable
by other instructors as instructors may have different skill levels with
the lesson content or differing mental models. In general, it is not a
good idea to assume others, learners or instructors, will know what you
were thinking when you wrote the content so, if in doubt, be
explicit.
Exercise: Examples Before Exercises (20 minutes)
Looking back at one of the exercises you designed before: what
examples could you include in your narrative to teach learners the
skills they will need to apply to complete the formative assessments you
have designed?
Outline one of these examples in your lesson design notes.
In the Software Carpentry Plotting and Programming with Python Lesson
there is an exercise to load and inspect CSV file for Americas. In the
lesson, before the exercise, the Instructor demonstrates an example of
how to load the data table for another continent (Oceania) and explores
the values with a few different functions. This shows learners how to
call the function to load the CSV into a data frame, and demonstrates
what success looks like for their exercise.
In the Python Interactive Data Visualization Lesson in the Incubator,
there is an exercise to find the correct widget (a slider) for an action
and modify the script to use it. In the lesson, before the exercise, the
instructor introduces a cheatsheet and documentation for interactive
widgets and then creates a dropdown widget for the application. The
slider widget required in the exercise has not been demonstrated but the
preceding example shows all of the necessary steps to add a widget, and
provides the supporting information that learners can consult to
discover how to implement the specific tool.
Less is More
Trying to fit too much content into a lesson is counter-productive.
It is better to cover less and provide a smaller but stable foundation
for learners to build upon.
You should also consider a realistic workshop length for teaching
your lesson. It is difficult to keep learners attentive for many hours
over many days. Most Carpentries workshops are two work days of lesson
material as it is difficult for learners to attend 4 full days of a
workshop. Although, there are some in the community who will run
workshops that are three days to cover additional material.
As you consider the length of your lesson discuss with your
collaborators and ask yourself:
What is essential to include?
What can be left out if needed?
Are there checkpoints where the lesson could end if needed?
Can important concepts be moved up earlier to ensure they are
covered?
In the end, the only way to know for sure is to teach the lesson,
measuring how long it takes to teach. Borrowing from television, The
Carpentries community calls these early workshops, pilot workshops. As
you run pilot workshops, you can note the length of time spent in each
episode. You might find the template
for notes on pilot workshops helpful as it includes a table for
episode and exercise timings. Instructors commonly report running short
on time in workshops. Thus, it is better to assume that you are
under-estimating rather than over-estimating the length. It is better to
have additional time for discussion, review, and wrap-up than to rush
through material to fit it into the remaining time.
If, after piloting your new lesson, you find that you did not have
time to cover all the content, approach cutting down the lesson by
identifying which learning objectives to remove. Then take out the
objective(s) and the corresponding assessment(s) and explanatory
content. You may consider making a concept
map to help identify which objectives depend on one another.
Alternatively, if you decide to keep certain objectives in the
lesson, you can add suggestions on which objectives can be skipped to
the Instructor Notes for the lesson. Instructor Notes are meant to
convey teaching tips and advice to other instructors teaching your
lesson and will be discussed in more detail later on in section “Preparing to Teach”.
As you add notes and think about what to cut, remember, reducing the
number of lesson objectives can help with managing learner (and
instructor) cognitive load.
Testimonial
“the five ways to handle an extraneous overload situation are
to:
eliminate extraneous material (coherence principle),
insert signals emphasizing the essential material (signaling
principle),
eliminate redundant printed text (redundancy
principle),
place printed text next to corresponding parts of graphics
(spatial contiguity principle), and
eliminate the need to hold essential material in working memory
for long periods of time (temporal contiguity principle).”
(5 minutes) In the shared notes document, note down your answers to
these questions:
From a design perspective, at what point is a lesson too long?
What factors influence and constrain the length of a lesson?
How might you prioritise what to keep if you have to cut lesson
content down?
(5 minutes) In the remaining time, your Trainers will lead a
discussion based on the responses.
To get started on the discussion part, try reading out an interesting
response and asking for more detail from the trainee who wrote it. Ask
one of the other trainers/helpers to take notes of this discussion in
the collaborative note-taking document.
This is a good opportunity to remind trainees about how
counter-productive it can be to try to cover more content than the time
allows. (See the earlier
section on lesson scope.)
Other Important Considerations for Lesson Content
Language
When writing your lesson text you also want to avoid unintentionally
demotivating your learners. Make time to review your text for:
use of stereotypes
(check learner profiles for stereotypes too)
expert
awareness gaps, i.e. places where you may be assuming the learners
know more than they actually do
use of different terms or notations with the same meaning
interchangeably without explanation, e.g. using the terms “shell”,
“bash”, “terminal” and “console” to describe the command line interface,
or “variables” and “columns” for column headings in data tables
unexplained or unnecessary jargon/terminology (as your learners may
come from different backgrounds, may be novices, not native English
speakers, and a term in one domain/topic may mean something else
entirely in another)
unexplained assumptions
sudden jumps in difficulty/complexity
Accessibility
You should also review your text thinking about accessibility. This
includes:
Avoiding regional/cultural references and idioms that will not
translate across borders/cultures
Avoiding contractions i.e. don’t, can’t, won’t etc.
Optional Exercise: Alternative Text for Images (5 minutes)
Which of the following is a good alt-text option for the image
below?
Atmospheric Carbon Dioxide at Mauna Loa
Observatory
Graph of data
Graph with increasing lines
Line graph of increasing carbon dioxide in ppm at the Mauna Loa
Observatory from 1958 to present
Line graph of increasing carbon dioxide in ppm at the Mauna Loa
Observatory, Hawaii, United States, from 1959 to present including
values from each year. Red line shows variation in each year and black
line is average for each year. 1959 = 315.90 ppm, 1960 = 316.91, 1961 =
317.64 …
The third option, “Line graph of increasing carbon dioxide in ppm at
the Mauna Loa Observatory from 1958 to present”, is the best option. It
describes the type of plot, what is measured, and the trend.
The first two options are too vague. They mention a graph but don’t
give enough info to know what the graph is actually showing.
The last option is overly descriptive and starts to list the data
itself. It is better to include a shorter, but informative, description
and a link to the data instead.
May want to point out that the answers in the current version of this
exercise do not have diagnostic power.
Glossary of Terms
When writing a lesson, you should also consider adding key terms or
jargon to the lesson glossary for the lesson. In your shared notes
document, you can start such a glossary as you develop your lesson, to
keep a record of the terminology your audience needs to be familiar
with.
Many of these terms may also be useful for other lessons and can be
added to Glosario, a
multilingual glossary for computing and data science terms.
Exercise: Explain Your Terminology (5 minutes)
In your shared notes document, add a list of terms or jargon from
your lesson, along with their definitions.
You do not have to have a complete list now, but it is good to start
working on it - you can always come back and update it later.
Questions and Keypoints
In addition to objectives, a completed
episode also requires questions the episode answers and the
keypoints a lesson covers.
The questions helps learners understand what to expect from a lesson
as they might not yet understand the learning objectives.
The keypoints wrap up the lesson by providing answers to each of the
questions. Keypoints also help self-directed learners review what they
learned, remind instructors to recap what was covered in an episode, and
help you ensure you answered all the questions you intended for an
episode.
In your shared notes document, add key points and questions for your
episode.
Polishing a lesson
Lessons do not need to be perfected prior to teaching them the first
time. In fact, keeping things relatively basic leaves you some
flexibility while you run through the first few iterations of teaching
the lesson, collecting feedback, and using that to guide
improvements.
However, once you find that the lesson is beginning to reach
something like a stable state, you may find the
checklist for lesson reviewers in The Carpentries Lab useful as a
guide for polishing the design and content. The checklist describes
criteria for a lesson to meet a high standard in terms of its
accessibility, design, content, and supporting information.
Key Points
The objectives and assessments provide a good outline for an episode
and then the text fills in the gaps to support anyone learning or
teaching from the lesson.
It is important to review your lesson for demotivating language,
cognitive load, and accessibility.
To reduce cognitive load and ensure there is enough time for for the
materials, consider which lesson objectives are not needed and remove
related content and assessments.
How is a lesson site set up and published with GitHub?
What are the essential configuration steps for a new lesson?
Objectives
After completing this episode, participants should be able to…
Identify the key tools used in The Carpentries lesson
infrastructure.
Complete the fundamental setup steps for a new lesson
repository.
Edit Markdown files using the GitHub web interface.
At this stage in the training, you will have gone through the first
three stages of the lesson design. You should have a clear idea of who
the people are that you want to teach this lesson to, and what exactly
the skills are that you want them to learn while they are following it.
You also have an outline of your lesson and its episodes. It is now time
to begin creating a website that will present that lesson to the
world!
GitHub Pages
The source of all The Carpentries lessons is made publicly-available
in repositories on GitHub. By making
our repositories public like this, we encourage others to help us
maintain and improve our lessons, and make it as easy as possible for
them to re-use and modify our lessons for their own purposes.
GitHub provides a hosting service to open source projects such as
these, allowing users to present their projects to the wider world. The
repository includes a complete history of the changes made between
versions of the individual files in the project, and provides many
features that facilitate collaboration on projects. We will learn more
about some of those collaborative features later in this training. For
now, we will focus on one other important feature that GitHub provides:
website hosting.
Via a system called GitHub Pages, users can build
and host websites from the files present in any repository on GitHub.
For many years, this has been how The Carpentries presents its lesson
websites to the world.
Using The Carpentries Workbench
Carpentries lesson websites are built with The Carpentries
Workbench, a toolkit that converts Markdown and R Markdown
files into the HTML that can be served by GitHub Pages. We will use it
now to initialise a new lesson.
In the past, our lesson sites were generated by software called
Jekyll, a tool built into
GitHub Pages that allows the webpage content, written in the text files
of the repository, to be combined with descriptions of settings,
structure, and styling, to create a website. The template used by Jekyll
to structure and style lessons was initially developed in 2013/2014 by
Abby Cabunoc Mayes, Greg Wilson, Jon Pipitone, and Michael Hansen for Software Carpentry. It was expanded
and maintained by many members of the community over almost a
decade, to also support Data
Carpentry, Library
Carpentry, and many other lessons.
In 2022, we adopted a new infrastructure for our lesson sites:
The Carpentries Workbench. Lesson sites built on the
Workbench are still hosted with GitHub Pages, but no longer use Jekyll.
Instead, the lessons are built using a programming language,
R, and pandoc, a
software designed for converting content between file formats. The
Workbench combines three R packages:
sandpaper: converts a collection of Markdown or R
Markdown files into the structure of a lesson website.
varnish: provides the files and folders that add
styling and additional functionality to a lesson website.
pegboard: a programmatic interface to the lesson,
enabling various automated validation tasks.
For lesson developers, the Workbench makes The Carpentries lesson
repositories much simpler to navigate and work with.
Developing lessons using the Carpentries Workbench in GitHub might
seem intimidating, especially for those who are less experienced with
collaborative GitHub workflows or unfamiliar with the Workbench.
However, most of the complex infrastructure is hidden from lesson
developers, and some setup tasks only need to be completed once or
infrequently. The table below outlines common tasks that lesson
developers perform in GitHub (which you will learn about shortly) along
with how often they are needed.
Task
Frequency
Create repository
Once
Add collaborators
Rarely
Edit global lesson configuration file
Rarely
Edit lesson front page
Rarely
Create a new episode
Often in the early stages of lesson design, rarely after that
Edit episode content
Often
Creating a Lesson Repository
To get started, we first need to create a new repository for our
lesson. We will use a template to do this, so that the new repository
contains the basic files and folders that the Workbench needs in order
to build a lesson site. There are currently two templates to choose
between:
An
R Markdown template, best suited to lessons you expect to include R
source code that will generate output.
One member of each participating lesson team should
choose one of these templates, following the link above and completing
the configuration as follows:
click on the green “Use this template” button near the top right of
the window
add a name for the repository
The name should be descriptive but fairly brief, with hyphens
(-) to separate words
the name can always be changed later, via the repository
settings
make sure the “Include all branches” box is checked to copy all
branches from the template repository
this is important as lessons are developed in branch
main but rendered into websites from branch
gh-pages
in the “Description” field, write the title of your lesson
choose “Public” visibility
After pressing the Create repository button, you should be
presented with a brand new lesson repository, like in the picture
below.
Directory structure of new lesson repository
created from a lesson template. Note that new repositories created from
the R Markdown lesson template will include an additional
renv/ directory.
Remind learners to paste URL of their lesson repository and the
rendered version of the lesson to the collaborative document.
Adding collaborators
To be able to add content to the lesson, your collaborators on this
project will need access to the repository. To add collaborators to the
repository, navigate to Settings, then choose
Collaborators from the left sidebar. Now repeat the following
steps for every collaborator working with us on the project.
Click Add people and enter the GitHub username of one of
our collaborators.
Click Add USERNAME to this repository.
Your collaborator should receive an email inviting them to join the
repository. After they have accepted this invitation, they should be
able to edit the repository, adding new files and modifying existing
ones. Only the person who created the repository will be able to adjust
the repository settings.
Note, the above directions assume your repository is owned by a
personal account not an organisation account. The options for
permissions are different when adding collaborators for an
organisational account. On a personal account, you only have two
permission options “owner” or “collaborator”, where organisations have
more roles with varied permissions.
The repository contains a number of files and folders. Most of these
are source files for the content of our new lesson, but a few are
accompanying files primarily intended for the repository itself rather
than the lesson website. These are:
CITATION.cff
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE.md
README.md
.gitignore
and the .github/ folder.
We will address all of the files later in the training. For now, we
will move on to complete the basic setup of the lesson.
Configuring a Lesson Repository
Since your lesson was created from a lesson template which contains
gh-pages branch, GitHub Pages should already be generating
a rendered version of your lesson from this branch. If it isn’t, the
instructor should work with groups to activate GitHub Pages.
We need to tell GitHub to begin serving the lesson website via GitHub
Pages. To do this, navigate to Settings, then choose
Pages from the left sidebar. Under Source, choose the
gh-pages branch, leave the folder set to
/ (root), and click Save. You may copy the URL in
this box, this will be the address of your lesson site.
Although we configure GitHub Pages to serve the lesson website from
the gh-pages branch, the default working branch for
a lesson will be main. For the rest of this
training, you should add and edit files on main, and in
future, when you open Pull Requests to update the lesson content, these
should also be made to main. The gh-pages
branch should never be modified by anyone other than the automated
actions-user account.
To see the rendering of the gh-pages branch by the
automated actions-user, you can click the “Actions” tab from the main
repo page. This tab shows the actions triggered by each commit and if
they are in-progress or completed (sucessfully or with an error). You
can also see the details of actions in more depth by clicking on an
individual commit in this page.
On the repository home page (e.g. by clicking the name of the project
near the top left of the window), click the gear wheel icon near the top
right, to edit the About box. Check the box that says “Use your
GitHub Pages website” to add the address of your lesson site to the
About box.
After following these steps, when you navigate to the pages URL, you
should see a lesson website with The Carpentries logo and “Lesson Title”
in the top-left corner. You may need to wait a few minutes for the
website to be generated.
For trainees unfamiliar with the relationship between a source
repository and rendered website, it is important to show the default
version of the lesson website now i.e. before beginning to edit
config.yaml.
It is also recommended to take time at this point to demonstrate
step-by-step how to edit files in the GitHub web interface.
config.yaml
The lesson title can be adjusted by modifying the
config.yaml file in the repository. The
config.yaml file contains several global parameters for a
lesson - to determine some of the page styling, contact details for the
lesson, etc. config.yaml is written in YAML, a structured file format of
key-value pairs in the form key: value. For example, a YAML
file of personal data might include lines such as
name: 'Mei', height_m: 1.84, and
birthdate: 1899-01-12. As well as the title of the lesson,
you can and should adjust some of the other values in
config.yaml, but you should not need to add new values or
learn a lot about YAML to be able to configure your lesson. You should
also only have to add or modify single-line string values and list
entries, and not deal with multi-line strings or other data types
(e.g. numbers, booleans, dictionaries) that YAML supports. The template
config.yaml aims to guide users through examples and
annotations on how to format the values they provide e.g. if you are
modifying a value wrapped in quotation marks in the template file, it is
safest to replace it with another value within quotation marks.
Practice with config.yaml (5
minutes)
Complete the configuration of your lesson by adjusting the following
fields in config.yaml. Remember that FIXME values with
quotes should be replaced with another value still in quotes and FIXMEs
without quotes should be replaced by values without quotes.
contact: add an email address people can contact with
questions about the lesson/project.
created: the date the lesson was created (today’s date)
in YYYY-MM-DD format.
keywords: a (short) comma-separated list of keywords
for the lesson, which can help people find your lesson when searching
for resources online. At a minimum, include ‘lesson’, ‘pre-alpha’, and
your lesson’s (human) language. For example, ‘lesson, pre-alpha,
español’.
source: change this to the URL for your lesson
repository.
We will revisit the life_cycle and
carpentry fields in config.yaml later in this
training.
.Rproj file
The repository also contains a file called FIXME.Rproj,
a project file that RStudio (a commonly-used editor for working on
lessons) can use to track some options and preferences for working on
your lesson on your local machine. This file should be renamed. You can
call this file whatever you like, but the convention is for its name to
match that of the lesson repository.
README.md
The README.md file is the “front page” of your lesson repository on
GitHub, and is written in Markdown. You should use it to provide basic
information about the project, for anyone who finds their way to the
source files for your lesson.
Improving the README.md (5
minutes)
Take a few minutes to update it with some basic information about the
project:
the lesson title
a short description of the lesson
a list of the names of the authors, optionally linked to their
GitHub profile
We will revisit README.md later in the training with
more details on what to include in this file.
Key Points
Lesson sites are built with GitHub Pages from source repositories
located in GitHub.
A new lesson repository can be created from a template maintained by
The Carpentries, and configured by adjusting the
config.yaml file.
How do you create and modify the pages in a lesson?
How should different structural elements be presented in a lesson
website?
Objectives
After completing this episode, participants should be able to…
Add lesson episodes as individual pages of a lesson website.
Use fenced divs to create different structural elements
within a page to format objectives, questions, keypoints, exercises and
their solutions in a lesson website.
Now that we have completed the basic configuration of a lesson site,
it is time to move on and look at where the actual lesson content will
be written.
As we have seen previously, lesson content is composed using Markdown
(or R Markdown), a lightweight markup language that simplifies the
formatting of plain text. While Markdown provides structured elements
like headings, lists, links, images, and code blocks, the Workbench
enhances this by allowing the creation of additional structured text
blocks that visually stand out from standard paragraphs on lesson
websites. These elements, known as fenced divs, will be
demonstrated shortly.
Let’s start by writing our lesson’s home page.
Lesson Home Page
The “home page” of a lesson is created from the index.md
file: this file should contain a brief introduction to the lesson,
designed to give visitors to the page a first impression of the lesson
and whether it is appropriate for them. The file begins with a short
header, written in the same key-value pair syntax we encountered in
config.yaml.
MARKDOWN
---site: sandpaper::sandpaper_site---
This header configures how the site will be built by
sandpaper, one of the components of The Carpentries
Workbench, and should be left untouched. The page content begins after
the blank line that follows this header.
To get started on this home page, let’s replace the template text in
index.md with the same short description of your lesson you
added to your README.md.
After that, it might be a good idea to add your lesson objectives to
our lesson’s homepage.
Exercise: more practice editing Markdown in GitHub
Add the objectives you defined for your lesson as a bullet list in
the index.md file of your lesson repository.
Below those, let’s add the prerequisite skills you determined earlier
for your lesson. While you can add them as a bullet point list, the
Workbench provides a special formatted block for prerequisites, so they
appear visually distinct on the page.
Fenced divs
Fenced divs are structural elements within the page, which get
rendered in a visually distinct way on the lesson website. Fenced divs
include regular Markdown content surrounded by special tags to mark the
start and the end of such a block.
To mark the beginning of a fenced div, start a line with an opening
tag - at least 3 colon characters followed by a blank character and a
keyword denoting the type of a fenced div you are creating
(::: fenced_div_keyword) . Then you add the Markdown
content of this structural block. Finally, end your fenced div with a
closing tag consisting of at least 3 colons (:::).
Once your page builds, check how prerequisites block appears on your
lesson website. A screenshot below provides an example.
A rendered lesson prerequisite fenced div
You might have noticed by now that GitHub provides a preview of
content in Markdown files, and displays an interpreted (“rendered”)
version of Markdown file content by default in its web interface. You
might also notice that the content of index.md is displayed
differently when viewed on GitHub from how it appears on the homepage of
your lesson website: the yaml header mentioned above appears in a
tabular layout on GitHub, but is not visible on the lesson homepage.
This is because the lesson infrastructure accepts an expanded set of
elements in the source Markdown files: syntax for components of a lesson
website that are not part of the standard set of Markdown features,
e.g. page metadata such as the YAML header above, or fenced div blocks
to make certain content visually distinctive. In these cases, you may
see GitHub render the content differently from how it appears on your
website, or not at all.
We will encounter more examples of this throughout the lesson, but
the important thing to remember for now is that how GitHub
displays source (R) Markdown files is not a reliable indicator of how
content will be displayed in your website.
Episodes
The main body of the lesson is written in episodes: the
individual chunks or sections that the lesson is separated into. The
episode pages of the lesson site will be constructed from Markdown or R
Markdown files in the episodes folder of the lesson
repository.
The episodes folder of the new repository contains a
single example episode file, introduction.md. The content
of this file includes some help with and examples of how to write
Markdown files for The Carpentries Workbench. Like
index.md, the episode file begins with a short header. The
fields in this header describe the episode title and the estimated time
(in minutes) required to teach it and complete the exercises.
Creating a new episode
Let’s create a new episode file, for one of the episodes you have
identified earlier in the training. First, open the “raw” view of the
introduction.md example episode, and copy its contents to
make it easier to structure the new episode we are creating.
You may ask each trainee in a lesson team to create an episode file
for the episode they are responsible for. Be careful here to ensure that
participants who are collaborating on the same repository do not create
conflicts e.g. by editing the same file or creating files with identical
names.
MARKDOWN
---title: "Intro to Markdown"teaching: 10exercises: 2---:::::::::::::::::::::::::::::::::::::: questions - How do you write a lesson using Markdown and `{sandpaper}`?::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: objectives- Explain how to use markdown with The Carpentries Workbench- Demonstrate how to include pieces of code, figures, and nested challenge blocks::::::::::::::::::::::::::::::::::::::::::::::::## IntroductionThis is a lesson created via The Carpentries Workbench. It is written in[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and[R Markdown][r-markdown] for dynamic files that can render code into output. Please refer to the [Introduction to The Carpentries Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation....::::::::::::::::::::::::::::::::::::: keypoints - Use `.md` files for episodes when you want static content- Use `.Rmd` files for episodes when you need to generate output- Run `sandpaper::check_lesson()` to identify any issues with your lesson- Run `sandpaper::build_lesson()` to preview your lesson locally::::::::::::::::::::::::::::::::::::::::::::::::
You may have noticed a few more fenced divs in the episode file -
questions, objectives and
keypoints. These are all compulsory elements of every
episode - the Workbench will refuse to build your lesson if any of these
are missing.
Now create a new file in the episodes folder. Based on
the episodes you planned out earlier, choose a name for your episode
that concisely describes the intended content,
e.g. data-visualisation.md (or
data-visualisation.Rmd for a lesson based on R Markdown).
It is vital to include the file extension when naming this file: only
files with the .md or .Rmd extensions will be
built into webpages by the lesson infrastructure.
For the content of your episode, paste everything from the
introduction.md file you copied earlier and:
replace the title in the header with the title of your episode
set the teaching and exercises fields to
zero for now
replace the contents of the questions div with the
questions for your episode you defined earlier
replace the contents of the objectives div with the
episode-level objectives you defined earlier
replace the contents of the keypoints div with the key
points for your episode you defined earlier
Ignore or delete the rest of the content of the episode (i.e. only
focus on the three structural elements - questions, objectives and
keypoints).
Adding a new episode to the lesson navigation
This new episode will not yet appear in the navigation of your lesson
site. To enable this, we need to specify where the episode should appear
in the order of the lesson. That episode order is defined in the
episodes field of the config.yaml file:
YAML
# Order of episodes in your lessonepisodes:- introduction.md
Add the name of the new episode file you created to this list in
config.yaml and commit this change, for example:
YAML
# Order of episodes in your lessonepisodes:- introduction.md- data-visualisation.md
After the lesson site has been rebuilt on GitHub, you should see the
episode title appear under EPISODES in the left sidebar
navigation of your lesson site after refreshing the webpage. Clicking on
that title will take you to the episode page built from the file you
created. At the top of the page body, you will find the episode title
and an Overview box containing a list of the questions and
objectives defined for the episode. At the bottom of the page, you will
find the keypoints. Later, we will add more content to your chosen
episodes.
An example rendered lesson episode page
Exercise: practice creating episodes (10 minutes)
Repeat the steps you just saw, to create another new episode file and
add it to the lesson site navigation. If you know what another episode
in your lesson will be about, create the page for that. Otherwise, feel
free to use any values you like for the file name and episode title.
Using this approach, we can build up our lesson one episode at a
time.
Adding Exercises
To create an exercise in The Carpentries Workbench, we use two types
of fenced div:
challenge, where the main task is a problem to be
solved.
discussion, where the main task is for participants to
discuss a topic or prompt
To start a challenge fenced div, write an opening tag of
at least 3 colons as usual, then follow it with the
challenge keyword. Add the content of the challenge on the
following lines. Finally, close the fenced div using another line with
least 3 colons.
MARKDOWN
:::::::::::::::::::::::::::::::::::::: challenge### Challenge TitleChallenge text, code, and other information goes here::::::::::::::::::::::::::::::::::::::::::::::::::
If you also want to include an expandable solution box for the
challenge you can nest a solution fenced div within the
challenge div. The format is the same as for a challenge
except the fenced div tag is solution instead. Note the
solutions can all be expanded for more accessible reading using the
“Expand All Solutions” option at the top of each episode.
:::::::::::::::::::::::::::::::::::::: challenge
### Challenge Title
Challenge text, code, and other information goes here
:::::::::::::: solution
### Solution Title
Solution text, code, and other information
:::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::
For readability, you may want to have the length of the closing lines
match the opening lines. See in the above how the challenge and the
nested solution’s closing lines are similar lengths to the their
corresponding opening lines. For more information about creating
exercises see the Workbench
Documentation for Exercises.
Exercise: Formatting Exercises in a Lesson Site (15 minutes)
Using the approach demonstrated above, format the exercise you
designed previously as an exercise in your lesson site.
Optional and Important Material
The Carpentries Workbench includes fenced divs for highlighting key
material that should not be skipped during instruction (with the
callout keyword), as well as optional content that can be
covered if time permits (spoiler keyword).
Often lessons have more content than can be reasonably taught in the
amount of time allotted. This is especially true for collaboratively
developed lessons as each contributor/instructor may have additional
items they’d like to see included, leading to “scope creep”.
To address this issue, in many Carpentries lessons,
spoiler boxes are used for asides and short tangents,
e.g. points that might be relevant to some audiences but are not
essential to the flow of the lesson. These spoilers should still be kept
to a minimum as they can be disruptive to instructors and readers.
Note, this use of the spoiler box is to use the expandable box
functionality as nothing will be “spoiled” by expanding this box. Also,
note that the spoiler box titles should be very clear so instructors
need not expand the spoiler to know if they want to teach that
extra.
Calling Attention to Important Points
For important points in the lesson, you can add them to a
callout box to emphasize their importance.
You may notice that many of the older lessons use callouts for both
additional material and to highlight important points
interchangeably.
Spoilers were developed in 2023 to help with separating these two
different use cases.
The Workbench also supports a caution class of fenced
dev, which can be used to draw attention to common pitfalls or otherwise
emphasise something that learners should be careful about.
Glossary of Terms
You have already started on compiling a glossary of terms for your
lesson. The Workbench offers a standardized location for lesson
terminology: learners/reference.md.
More Practice with Lesson Content
Exercise: More Practice (15 minutes)
Use this time to add more content to your lesson site. Generally
speaking, you should try to transfer the drafted content from your
lesson design notes into the lesson website. Here are some suggestions
for things you might try:
Add other exercises - e.g. add a discussion. Note that
the lesson infrastructure does not support solution divs
attached to discussions.
Look through the Workbench
component guide and try implementing some of the other flavours of
fenced div.
Add a new Markdown file to the learners/ or
instructors/ folder and see if you can find the built page
in your lesson site.
Troubleshooting the Lesson Build
This is a good opportunity to pause and check in on how well
trainees’ lesson builds are running. If anyone is having trouble with
their workflows, ask them to share their screen and try following the
troubleshooting steps to diagnose and fix the issue.
This is also a good opportunity to show what the GitHub actions look
like when they are in progress, succeed, or fail if you haven’t
already.
If all is well, and you are pressed for time, this section can be
skipped.
Sometimes, formatting errors and typos in the files of your
repository can cause the process that builds your lesson website to
fail.
You will likely first notice the failure to build the lesson if the
lesson website is not showing the changes you made. You may also receive
an email from GitHub about the build failure if that is your preferred
way of receiving notifications (but it may take you some time to realise
this). The good thing is that GitHub keeps your previous lesson version
online until the error is fixed and a new build is completed
successfully. If you are not yet familiar with the GitHub interface, it
can be difficult to pinpoint the problem.
Here are some points of advice that you can follow to help find and
fix the problem when you notice the build process fail.
When you first notice that there is a problem, stop
editing your lesson: any subsequent changes that you make will not be
included in your lesson site until the problem is fixed, and might
introduce additional issues, making it more difficult to find the
original cause.
Look at the history of commits (the link with a
reversing stopwatch icon and “NN commits” at the top of the listing of
files and folders in the repository homepage). Is there a place where
the build has a red cross instead of green circle? If so, click on that
commit to look at the “diff” (where it shows which lines have been
modified). The error is likely to have been introduced by these
changes.
Ask for help: you can open an issue on your
repository and tag a member of The Carpentries team to ask for help
finding the problem. Alternatively, you can also ask for help in The
Carpentries Slack workspace. (More about communication channels for
discussing lesson development coming later in this training.)
Waiting for the GitHub Actions process that builds the website to
run can be tedious, and slow down the process of troubleshooting: it can
take a few minutes for the process to reach the point where it
encounters the problem. It can be helpful to download the files for your
lesson and try to build a version of the website on your local
computer, where the build process will be much faster. The
Workbench documentation provides instructions
for installing the infrastructure and building
a local preview of the lesson website.
Keeping your infrastructure up to date
New versions of the workflows that build your lesson site will be
released occasionally, and the Workbench includes a maintenance workflow
that can help you keep your infrastructre healthy. If you are using R
Markdown source files in your lesson, you will also need to update the
package dependencies of your lesson and the Workbench provides a second
maintenance workflow that can take care of that for you too. Both of
these maintenance workflows will open pull requests on your lesson
repository, which you will need to merge for the updates to be applied.
If your lesson is hosted in The Carpentries Incubator, these workflows
will be configured for you. If you are keeping your lesson repository
somewhere else instead (e.g. with your personal GitHub account or an
organisation of your own), you will need to take care of a
few configuration steps to allow these maintenance workflows to open
pull requests on the project on your behalf.
Key Points
The main pages of a lesson website (lesson episodes) are created
from individual Markdown or R Markdown files in the
episodes folder of a lesson repository
Objectives, questions, keypoints, exercises (and solutions), and
other “special” structural page elements (other than plain text) can be
formatted using fenced div blocks - they are rendered visually
differently in the lesson website.
There are many types of fenced
divs available to lesson developers.
What are the important milestones in the development of a new
lesson?
How can The Carpentries lesson development community help me
complete my lesson?
Objectives
After completing this episode, participants should be able to…
Describe the role that feedback plays in the life cycle of a
lesson.
Connect with other members of the community.
Until now, this training has focused on the internals of a lesson
development project. Now it is time to consider the evolution of a
lesson over time, and the context in which lesson development can take
place.
The Lesson Life Cycle Revisited
At the beginning of this training, you were introduced to the concept
of a lesson life cycle, which The Carpentries promotes to help
developers communicate the status of an open source lesson project to
instructors and learners.
The life cycle of a lesson, annotated to
indicate the platforms provided for lesson projects at each stage of the
cycle.
Indicating Your Lesson’s Progress
The life cycle stage of a lesson is displayed as a banner on the
lesson site, as a label on the lesson GitHub repository, and alongside
the lesson whenever it is shown on The Carpentries websites and lesson
listings.
The life cycle stage for a lesson is configured in the
config.yaml file we encountered when we were first
introduced to The Carpentries
Workbench, as a value for the life_cycle field. For new
lesson repositories, this value is already set to ‘pre-alpha’, so you
should not have to change it yet.
The config.yaml file also contains a
carpentry field, which can be used to adjust the styling
applied to a lesson website e.g. to make it look like a lesson from the
Software, Library, or Data Carpentry Lesson Programs.
For community-developed lessons, where no official relationship
exists with one of these Lesson Programs, you should keep using the
incubator setting, before potentially switching over to
another styling when the lesson moves e.g. into The Carpentries Lab.
Pathways out of Lesson Incubation
The Carpentries provides pathways for mature lessons to leave the
Incubator.
A mature lesson may join an existing lesson
program, e.g. Data Carpentry, Library Carpentry, or Software
Carpentry, subject to review and approval by the relevant Curriculum Advisory
Committee.
Developers can submit their lessons for open peer
review in The
Carpentries Lab, which hosts a growing collection of high-quality,
stable lessons created by the community. Developers submitting to the
Lab have the option of publishing their lesson in The Journal of Open Source Education
(JOSE).
A lesson does not need to be stable to be useful to the community:
lessons with alpha and beta status are already valuable resources to be
taught and reused.
For these pilot workshops to provide an effective evaluation of the
lesson, it is essential to seize the opportunity they provide to collect
feedback and data. For example, a pilot workshop offers a chance to
answer questions like:
How much time does it take to teach each section of the lesson?
How much time is required for each exercise?
What technical issues were encountered during the lesson?
What questions did learners ask during the workshop?
Which parts of the lesson were confusing for learners?
Which exercises could be improved to provide more information to the
instructors?
The
pilot workshop notes template provides a starting point, but it
takes a lot of time and effort to keep track of all this information
during a pilot workshop. It can be helpful to assign this task to a
member of the lesson development team, who can dedicate themselves to
observing the pilot and taking notes. As soon as the pilot has finished,
it is a good idea to take some time to share the notes with the other
lesson authors, to reflect on and discuss the experience of teaching the
lesson, and to synthesise the lesson notes and any action items that
come from this debrief into specific action items to improve the lesson.
These should be added as new issues on the lesson repository, to help
you keep track of the work that needs doing for the next iteration.
For beta pilot workshops, where the lesson is taught by instructors
who have not yet made a major contribution to its development, it is
vital that instructors have the opportunity before the workshop to learn
about the lesson from the authors, and to debrief with the authors
afterwards so that they can share their experience and observations
about how the lesson could be further improved.
The Carpentries
Incubator hosts a thriving community of lesson developers, working
on lesson projects at every stage of the life cycle.
You can add your lesson project(s) to the Incubator by submitting an
issue to the Incubator
Proposals repository. Lesson developers working in the Incubator
benefit from increased visibility for their projects and dedicated
support from The Carpentries team.
Connecting with the lesson developer community can be a great way to
find collaborators to contribute to or test your lesson, to stay up to
date with the latest support provided to the community, and to learn
from the experience of others.
Here are a community activities and channels that you might be
interested in joining:
The lesson-dev channel on The Carpentries Slack
workspace. This is a platform for the community to ask questions and
make announcements about lesson development. You could also browse the
other channels on the workspace for any that are relevant to the topic
of your lesson.
You may also find it helpful to create a new channel on Slack for
discussion of your lesson. Chat channels like this can be valuable ways
for a remote team to communicate and collaborate.
The
incubator-developers list on TopicBox. The Carpentries
Curriculum Team uses this mailing list to make relevant announcements to
the community of lesson developers working in Incubator.
The Carpentries Curriculum Team hosts Lesson Development Coworking
Session on occasion, which are a good opportunity to engage with other
lesson developers and make regular progress on your project. The
sessions are listed on The
Carpentries community calendar.
The Carpentries Workbench does not yet support multiple (human)
languages in a single lesson site, e.g. to see the lesson in Spanish and
Afrikaans as well as English. Nevertheless, a thriving subcommunity
exists of people translating/localising lesson content for other
languages and regions, making use of the CrowdIn tranlsation platform and tooling
developed by Joel Nitta that extends the Workbench. To connect with
the internationalisation (often abbreviated to i18n)
subcommunity, join the #internationalisation channel on The
Carpentries Slack workspace. Some of the groups translating lessons to a
particular language meet for regular coworking sessions. These events
are typically listed on the community calendar (linked above).
Exercise: join relevant channels (5 minutes)
Use this time to explore the options listed above and join/subscribe
to any communication channels that you find interesting.
Further Reading
The
Carpentries Lesson Developer Handbook provides guidance and a list
of resources for lesson developers to consult as they move through the
different stages of development.
Key Points
Teaching a lesson for the first time is an essential intermediate
step in the lesson development process.
The Carpentries lesson developer community shares their experience
on multiple communication channels.
What can I do to prepare to teach my lesson for the first time?
How should I communicate lesson setup instructions to learners?
What information should be recorded for instructors teaching a
lesson?
How should information be collected as part of the feedback
process?
Objectives
After completing this episode, participants should be able to…
Summarise lesson content as a teaching plan.
Add Setup Instructions and Instructor Notes to the lesson site.
Create a feedback collection plan.
In this episode, we will discuss how you can
measure learner progress and gather feedback about the effectiveness of
your content by teaching the lesson.
By now you should have developed objectives, exercises/formative
assessments, and examples for at least one of the episodes in your
lesson. In order to prepare to teach it (for the first time in
particular) you’ll need to create a teaching plan and design a method to
collect feedback on your lesson.
Teaching Plan
A teaching plan outlines the structure of your teaching session,
including details, estimated duration and materials needed to deliver
it. For example, a teaching plan may contain the following:
welcome, introductions and motivation for the lesson - to introduce
yourself, aims and objectives of the lesson and how it will help
learners
setup - to check if everyone is ready to proceed with the
lesson
segments of teaching and exercises from the lesson and any other
planned activities (e.g. group discussions, wrap-up and feedback)
checkpoints - places where you have planned to stop to check-in with
learners
slides, figures and other visual aids needed to deliver teaching and
other parts of the teaching plan
resources/references/recommended reading
Checkpoints in a Teaching Plan
Checkpoints are especially useful at the beginning of a lesson when
you may not have as many exercises but the learners need to have the
data downloaded or tool opened to be able to follow along. A checkpoint
could include stopping to ask what questions learners have or having
them indicate if they have completed the most recent step shown by the
instructor.
Perspective on Preparing a Teaching Plan
“I usually create detailed notes organised in a concept
map/workflow in order to make everything consistent. I also write next
to each bit of teaching the time approximately needed. For each lesson I
create (and reuse when I teach again the same lesson) very detailed
notes of what happens during the lesson, including teaching goals and
learning objectives I mean to achieve with each bit of lesson. This rich
lesson plan requires a lot of work the first time I teach a lesson, but
reduces a lot the preparation time of future lessons and are super
useful to other teachers.”
- Dr. Allegra Via, Carpentries Instructor and Instructor Trainer
Exercise: Prepare a teaching plan (15 minutes)
Create a bullet point list or brief notes describing what you will
say and do when teaching the episodes you have been focussing on during
this training.
You could create this teaching plan in a document just for yourself
or write it into your Lesson Design Notes document.
Figures and other visual aids are great tools to help participants
understand the content being presented and make associations between
pieces of information. The Instructor View of the lesson website
includes a link in the top menu to Extract All Images,
providing a single page containing all of the images in the lesson and
their captions: this can be useful as an alternative to a slide deck
when refering to figures and other visual aids during a workshop.
Setup Instructions
If your lesson requires participants to use any software, tools or
data, it is worthwhile investing time now into creating clear download,
installation and setup instructions for participants to follow and
circulating them ahead of the workshop. This will give you more time at
the pilot for teaching the lesson as troubleshooting setup problems on
the spot can be time-consuming and stressful. Each Carpentries lesson
has a dedicated place for setup instructions within The Carpentries
Workbench located in the file learners/setup.md off the
lesson project root.
Exercise: Add Setup Instructions (10 minutes)
Add setup instructions (in the learners/setup.md file)
with a list of software/tools/data needed by participants to follow your
lesson and links on how to obtain and install them.
Rather than producing a separate page in the lesson site, the
contents of learners/setup.md will be combined with
index.md to produce the Learner View of the landing page of
your lesson.
Instructor Notes
Another standard piece of The Carpentries Workbench are the
Instructor Notes pages - intended to help you and other
instructors deliver your lesson by providing useful tips for how to best
run a workshop or explain “sticky” points in the lesson. Instructor
Notes for your lesson are located in the
instructors/instructor-notes.md file off your lesson
project root - this page is displayed in the header menu when in
“Instructor View”.
You may consider adding the following information to your lesson’s
Instructor Notes:
rationale, strengths and/or weaknesses of the lesson design
what has been tried before and what worked/did not work
tips for teaching the lesson/particular episodes
challenges encountered
troubleshooting common setup/installation/exercise issues
parts that seem to cause trouble or most confusion for learners
(i.e. where to tread carefully, spend more time explaining)
You can also add notes in a specific episode of the lesson using
fenced div with tag “instructor”. Instructors can see such notes within
the lesson and the “Instructor Notes” tab in the header menu of the
website by changing the view in the upper right-hand corner of the
lesson to “Instructor View”. These notes are hidden when in ‘Learner
View’
It may initially be difficult to create Instructor Notes until you
have taught the lesson yourself for the first time - but after the first
pilot you will have enough feedback to draft them.
Exercise: Add Instructor Notes (5 minutes)
Add Instructor Notes (in the
instructors/instructor-notes.md file) with an initial list
of points that will help you and other instructors deliver the
lesson.
Add a single instructor note in one of your episodes using the
“instructor” fenced div.
Then try out the “Instructor View” of your lesson and seeing the
Instructor Notes in the header menu and the in-line hidden instructor
note in your episode.
The concept of hidden curriculum describes
curricular effects that influence learning but are not explicitly
addressed in the lesson content - e.g. (unofficial) social and cultural
norms, behaviours and values that are transferred by instructors to
learners, often unconsciously. These are all the additional things that
people learn about the topic from the way your lesson is
taught, rather than from its official content. It is worthwhile to spend
some time thinking about what learners could pick up from the
way your lesson is taught, as well as from its written content. You
can use Instructor Notes to guide instructors on some of the aspects of
the hidden curriculum of your lesson, e.g. to encourage them to follow
certain tried and tested practices.
Feedback Collection Plan
There is a lot to collect feedback on when trialling a new lesson and
this is arguably the most important part of your teaching plan. It can
be hard to capture all this information yourself while you are teaching,
hence you will need help from your team - make sure you assign the
feedback collection roles ahead of your pilot. Additionally, when you
are advertising your workshop, try and make it clear to the participants
you are piloting a new lesson and that they will be actively helping you
refine it as well as learning new things themselves. This can help focus
their attention on the feedback you need to collect from them, which you
should be doing:
constantly and throughout the workshop
As notes taken by an observer/co-instructor - refer to the “Pilot Workshops” episode for
suggestions on things to track and the pilot
workshop notes template and other suggested feedback collection
mechanisms to help you gather such information.
As “minute card” feedback from learners - minute cards are short
comments from learners on what is working well and what could be
improved, provided frequently (e.g. at the end of each session). Like
exercises used as formative assessment, minute card feedback can be
provided by learners while there is still time for instructors to act on
it and adjust their approach. You may find the
Virtual Minute Card template helpful for workshops run online, and
the
Instructor Training curriculum describes how sticky notes can be used at
in-person events.
at a designated wrap-up and feedback session at the end of
the workshop. Here, you can cover any other feedback that was
not addressed up to that point and any specific questions you may want
to ask your audience (e.g. about the choice of topics for the lesson,
difficulty level, workshop operations). You may design a special form
that participants can fill in anonymously - here is a post-workshop
survey template for workshops that pilot a lesson in The Carpentries
Incubator. You can use it as a starting point for a post-workshop survey
to be shared with your learners at the end of the pilot workshop and you
can supplement it with additional questions.
The final part of this training will focus on the skills needed to
collaborate effectively. Before that there will be a break, during which
we would like you to complete the following three tasks:
Teach one episode of your lesson (probably the one you have been
working on in these two days). See the Lesson
Trial Runs page for full details.
After your trial run has concluded (immediately after, or when you
have reviewed any feedback you collected from learners), note down your
answers to the following questions:
What worked?
What did not?
What will you do differently next time?
What will you change in your material you taught? We will refer to
these notes when we reconvene for the last episode of this
training.
Based on your experience teaching the material and the feedback you
received from your learners and helpers, make a list of issues you have
identified with the material you prepared, e.g.:
examples that did not work as expected,
improvements that could be made to exercises,
parts that learners found particularly challenging,
unexpected questions or misconceptions that came up during the trial
run.
We will return to these notes during the final training session, so
please make sure you save them somewhere you will be able to find them
again easily when the time comes.
Some feedback from participants that demonstrates the importance of
trial runs.
Testing with a ‘Real’ Audience
Presenting our lesson in front of a ‘real’ audience as a trial run
gave us valuable feedback on things we hadn’t fully considered in the
planning phase. It helped us to better understand how much time is
needed for the exercises, how difficult the exercises are for the
learners and how some examples used can be misunderstood by the
learners. I think that learner feedback is more useful the more the
audience closely matches your own target audience, so the insights are
more practical and relevant for improving the content.
- Julia Tolksdorf
Getting Feedback Early
Doing a trial run of our lesson so early in the process felt a bit
intimidating at first, it seemed like we weren’t ready and everything
needed more work. But that’s exactly why the trial run was so helpful.
It gave us a clear picture of what worked and what didn’t, like
realizing we’d packed in way too much content and spotting some
technical issues we hadn’t thought of before. The participants were
understanding and gave us really useful feedback, which made the next
version of the lesson so much better. It was a great reminder that
progress matters more than perfection.
- Olga Minaeva
What questions do you have? (15 minutes)
The homework from this workshop includes a trial run of one of the
episodes you have been developing in your lesson, to a real
audience.
After reading the information provided
about the trial run task and thinking about this task, what
questions do you have about how you should approach teaching that trial
run? Is there anything you are unsure of? What resources might help you
prepare for that experience?
Key Points
Spending time on preparing your teaching and feedback collection
will make you and your participants get the most out of your workshop
pilot.
Creating clear setup instructions as part of your lesson and
circulating them ahead of the pilot is time well-invested and will give
you more time when teaching the lesson.
Instructor Notes are teaching tips that you should include with your
lesson to help you (a few months down the line) and other instructors,
who have the relevant topic knowledge but have not been involved in the
lesson design and development, deliver your lesson more
successfully.
What needs to be done before the next part of the training?
Objectives
After completing this episode, participants should be able to…
Summarise the lesson development process introduced in this
training.
Describe the trial run task to be completed before the next part of
the training
Identify remaining tasks that should be completed before trialling
lesson content.
Recent sections of this training have focused on how to design
effective assessments and write a lesson that will be engaging and
motivating for learners. We provided further opportunities to apply
these concepts to the development of a lesson website with The
Carpentries Workbench. Next, we explored The Carpentries lesson
development ecosystem and how a new lesson project can fit into that. We
finished up with a discussion of the importance of teaching a new lesson
and how best to prepare for that experience.
The final sessions will focus on collaboration skills, exploring how
trainees can curate their repository and manage their project to foster
collaboration and build community around a lesson.
Before that, there will be an extended break during which trainees
should conduct a trial run of the lesson content they have been
developing in the workshop so far. When the training recommences, there
will be an opportunity for trainees to share their experiences and
reflections from those trial runs, and to discuss what impact that will
have on the next stages of their lesson development.
Polish a Lesson (20 minutes)
This exercise provides you with a chance to look back over everything
you have sketched out for your episode(s) and the lesson as a whole and
consider what still needs to be done before it can be taught.
You can use this time however you judge will be most beneficial to
your preparations for teaching your episode in a trial run.
If you are not sure how to start, consider the following prompts:
Which of the exercises set so far in this training did you not have
time to complete?
What do you still need to add/work on?
What can you remove/consider removing?
How will the narrative and example data you have chosen for your
lesson support teaching and assessment?
What diagram or other visual aids could you create/add to supplement
your text?
Please spend some time reflecting on what you learned so far and then
provide another round of feedback to your trainers.
Organise Your Knowledge (10-15 min)
Take some time to think back on what has been covered so far, then
make some notes on the most important points and actions you want to
take away from that. The Trainers and other participants will not look
at this - it is only for you.
If you do not know where to start, consider the following list for a
starting point:
draw a concept map, connecting the material
draw pictures or a comic depicting one or more of the concepts
write an outline of the topics we covered
write a paragraph or “journal” entry about your experience of the
training today
When the first draft of your lesson content is nearly complete,
consider mapping out the relationships between the objectives of your
episode and the examples and exercises via which they will be taught and
assessed. For example,
The read CSV and inspect demo supports Objective 2 (load a simple CSV
data set with Pandas) and will be delivered using participatory live
coding. The objective will be assessed with an exercise that requires
learners to apply the read_csv function to another file and count the
rows in the resulting DataFrame object.
Does any of your planned content not support any learning
objectives?
Is there at least one piece of content planned for each learning
objective?
Is there a formative assessment planned for each learning
objective?
Feedback (5 min)
The Trainer(s) will ask for feedback on the training so far. Take
some time to provide this feedback, before moving onto the second part
of this task.
Please stay in touch as you plan for your trial run.
If trainees would like to add the Design Notes document they have
been working on to their lesson site, they can do the following:
replace the first 13 lines with
---
title: Lesson Design Notes
---
save the file to the instructors/ directory, and any
images used in the file to the episodes/fig directory (the
paths to source files for images will need to be adjusted to begin
episodes/fig/)
# Information for Instructorsinstructors:- design-notes.md
This will make the page accessible from the ‘More’ dropdown in
Instructor View.
Note that concept maps with GraphViz are not currently supported
by the lesson infrastructure, so any concept maps added to the document
on CodiMD will not be displayed correctly on the lesson site.
Key Points
We have learned about the importance of exercises and data in a
lesson, the ecosystem The Carpentries provides for lesson development,
and how to prepare to teach a lesson for the first time.
Before the next part of the training, you should teach a part of
your new lesson to a real audience and reflect on the experience.
How to get in the queue to participate - hand-raising, typing in
chat, etc.
Microphones muted when you aren’t speaking (if virtual)
Any other expectations for interaction
Most of the episode is facilitated discussion. You may spend a lot of
time in the exercise that has them discuss the trial runs. You can
especially anticipate it taking more time when training more lesson
teams because they each need time to debrief and discuss with the whole
group. (See Trainer Note before the Trial Run Discussion exercise below
for further guidance.) Don’t panic if this happens! It has historically
been easier for Trainers to fit all of the content of this second part
of the training into the time available, so you can probably allow
yourself some extra time if you need it.
In this episode, we will discuss the final step
of the iterative lesson design process: how you can use the notes,
information, and feedback you collected when trialling your lesson to
identify ways that the design and content could be improved.
In part 2 of the training, we will learn more about the skills and
tools you can use to become an effective collaborator on an open source
lesson. Before moving on to cover that, we will spend some time
reflecting on and discussing the experience of teaching new lesson
material for the first time.
Exercise: Check in (10 minutes)
Re-introduce yourself and remind the group about the lesson you have
been working on.
In the break since last time, you should have trialled part of the
lesson you set out to develop in this training. Congratulations on
organising and delivering your first pilot! We will use this session to
reflect on that experience, and take some steps to ensure that the
feedback and notes you gathered during the trial run are used to improve
the quality of your lesson.
For online trainings, Trainers have found it beneficial to give
participants the first 5 minutes together in a breakout room, to
re-familiarise themselves with the notes from their Trial Runs and
discuss what they are going to report out to the rest of the group.
In trainings with a larger cohort of participants, try asking groups
to assign a reporter who will be responsible for summarising their
experience and answering the discussion questions on behalf of their
collaborators.
This exercise can take a long time when you have a lot of lesson
teams in your training, but it is important to give every group the
chance to reflect on their trial run. You may need to facilitate it
differently if time becomes an issue: for example, you could paste the
prompts into the CodiMD and ask all groups to write their answers,
before summarising responses, identifying trends, and calling on
particular trainees to share more about their individual responses.
Discussion: trial runs (35 minutes)
Look back at the notes you took and feedback you received during your
trial run (5 mins).
Now report out to the other participants about that trial run. Try to
answer some or all of the following questions:
What worked well both in terms of content and delivery?
What did not work as well?
How close were your time estimates to the actual time needed to
teach the material and finish exercises?
How did the audience perceive the difficulty level of the
material?
What will you do differently next time?
What will you change in the material you taught?
What will you change in the way you collect feedback in future
pilots?
Moving forward from your first pilot, the next steps to consider
are:
Record all the identified changes and improvements that can be made
to the lesson material and assign a person from the team responsible for
each task. The above exercise provided some reflection points after a
trial run; in terms of lesson content, they may translate into the
following considerations:
is the material too dense, does it need extra explanations?
would adding a diagram help explain or curate things better?
is there too much content for one episode, and do you need to split
it into smaller teaching units?
do you need to re-organise and move some content around to improve
the flow and narrative?
are there enough exercises and practical work?
do you need to realign your lesson objectives and key messages?
Decide if for some of the feedback you will not take action.
Remember, you do not need to respond to every piece of feedback you
receive. For example, it is easy to fall down the rabbit hole of adding
extra material just because someone suggested it may be a useful thing -
sometimes, adding a link or a spoiler pointing to extra reading is
sufficient. You need to be able to draw a line under any extra
modification suggestions to keep you in scope and on schedule.
Schedule follow-up co-working sessions with your team to carry on
working on fixing issues and adding new content to maintain the
momentum.
Add/Update your lessons’ Instructor Notes based on what you learned
and to help other instructors who will teach your lesson in the
future.
Think about the timeline for your next pilot(s), even provisionally,
to help you set milestones and targets to work towards.
Consider writing up your pilot experience in a mini blog post that
you can share with the community.
Designing and developing quality lessons is hard - there are many
pieces of a puzzle that have to come into place: both pedagogical and
organisational. Do not be disheartened with the amount and type of
feedback you may have received. Even Carpentries lessons that have been
around for 10+ years receive improvement suggestions and fixes almost
daily. From our experience, bigger lessons that are delivered over a few
days require several full pilots before they can even be considered for
a beta
release. Planning smaller lesson trials (where you test only a portion
of a lesson) and doing them more often with a friendly audience from
your local groups and close colleagues is more manageable and will help
you make steady progress.
Key Points
“Perfect
is the enemy of good” - your lesson does not need to be perfect
before you pilot or release it for community review. Early feedback from
the target audience will help you avoid straying off your lesson
plan.
Identify changes and improvements you want to make as a result of
trialling your lesson and schedule co-working sessions to work on these
tasks.
How can I use GitHub to manage (report, assign, track, prioritise)
tasks of a lesson development project?
What communication channels are effective for a lesson development
team?
How can I use GitHub to manage contributions from
collaborators?
Objectives
After completing this episode, participants should be able to…
Report and track issues with their lesson using the GitHub
interface.
Review and provide feedback on contributions from collaborators
using the GitHub interface.
In this episode we expand on how to use GitHub effectively among a
group of known collaborators, currently your lesson development group,
on an open source lesson development project, setting the stage for
introducing tools and practices for working with newcomers. Here, we
explore GitHub features to help you keep track of what needs doing on
the lesson, manage contributions and prioritise work. Making decisions
and managing your project as a whole will be covered in a follow-up
episode on project management.
Collaboration skills covered in this and follow-up episodes are
transferable to any similar open source project you may be involved
with.
Managing Issues
Issues are GitHub’s framework for managing issue/bug
reports, feature requests, and lists of future work. They provide a
single shared record of all the problems people have found with the
lesson, and improvements that could be made, along with solutions and
discussions around them. This helps the team to keep track of what they
need to work on later, and reduces the chance of receiving redundant
reports of issues you already know about.
When you create an issue, you can add a range of details to it. An
issue can be assigned to a specific team member - this can be a helpful
way to know who is currently working to fix an issue or a way to assign
responsibility. You can assign one or more labels to issues to classify
or group them. Labels can also be given colours, allowing you to see at
a glance from the Issues page the state of your project.
Lesson repositories created from the template are configured with The
Carpentries standard set of labels, which are designed to help
lesson developers and maintainers to indicate the type and current
status of issues. Some of the most frequently used labels include:
type:bug - indicates something that is broken in the
lesson, e.g. a factual error or broken link
type:enhancement - indicates that the issue is
proposing a way for the lesson to be improved
type:accessibility - indicates that the issue relates
to the accessibility of the lesson
status:wait - indicates that the issue cannot or will
not be worked on until something else is resolved, e.g. the solution to
the issue is dependent on another change yet to be made (see also:
status:blocked)
status:waiting for response - indicates that further
progress cannot be made until someone (usually the person who submitted
the issue) has responded
status:in progress - indicates that someone is working
on a resolution for the issue, preventing potentially redundant efforts
from being made by others
good first issue - indicates that the issue requires
relatively little knowledge of the project to fix, making it a good
choice for first-time contributors
help wanted - indicates that a maintainer wants help on
an issue or pull request
GitHub provides a default set of labels for repositories which can be
used “as is” or customised (as done by The Carpentries above) - check
the GitHub
documentation on Issues for the full reference. You can also create
your own custom labels to help with classifying issues.
If you are developing your lesson in The Carpentries Incubator,
you can register the repository for inclusion in The Carpentries Help Wanted Issues
page, which will list all issues labeled with
help wanted and/or good first issue. Issues
listed on this page are more visible to the wider Carpentries community,
making it a good way to attract spontaneous contributions from
newcomers.
Exercise: creating issues (15 minutes)
After your trial run, you should have created a list of issues you
identified with your lesson. Open issues on the GitHub repository for
your lesson, for each item on that list. Remember to add labels to each
issue according to its type, priority, etc. As you create them, you may
also want to assign these issues to a member of your lesson development
team.
Managing Contributions
Although it is possible (for collaborators with sufficient access to
the project) to make changes to a lesson directly on the
main branch of the repository, doing so is not
recommended:
It increases the likelihood of unintentional conflicts being created
when two collaborators work on the same file simultaneously.
It does not provide an opportunity for changes to be checked and
discussed before they are included in the lesson. This makes it much
more likely that mistakes will be included in the lesson, and that the
commit history of the project will include many changes made only to
fix/polish/undo previous commits.
It prevents your team from benefiting from the powerful features
that Git and GitHub provide to support and foster collaboration.
Instead, it is recommended practice to do most work in
branches: sets of changes that build on the work in the
main branch (or another) but are not included in that
branch until they are explicitly merged into it. GitHub
provides a way for this merging process to be managed, by giving a
platform for the changes to be described, discussed and reviewed, before
a decision is made to integrate or not integrate the changes in the
default branch of the project and the associated lesson. On GitHub,
these units of discussion, review, and acceptance/rejection of the
changes within a branch are called Pull Requests.
Pull requests are analogous to tracking changes in a shared online
document - both involve proposing, reviewing, discussing, and finalising
updates collaboratively. In a shared document, edits are submitted for
review by a proposer, akin to creating a pull request in GitHub, where
changes are proposed via a branch. Reviewers in both cases look at the
proposed changes, provide feedback via comments, (optionally) request
modifications, and may need to resolve conflicts between multiple
competing versions before final approval. Once approved, changes are
integrated into the main document (the main branch in your
lesson repsository), with version histories in both systems providing an
audit trail of contributions. This structured workflow ensures quality
and accountability in collaborative efforts.
The term Pull Request originates from the early days of
GitHub, when a contributor would be requesting that the project
maintainer(s) use Git to fetch (download) their branch and merge the
changes into the main branch of the project. In the terminology of Git,
this combination of fetching and merging a set of changes is called
pulling. The name Pull Request is slightly anachronistic now,
because a project maintainer can often manage the whole process of
reviewing a contribution in GitHub’s web interface.
Note: A common nickname for a Pull Request is PR or PRs for
Pull Requests.
Submitting and Reviewing Pull Requests on GitHub
The Trainers will collaborate on a demonstration of the process laid
out below.
Pull Request Activity
Note: This activity is designed to be carried out by two Trainers
acting in two different roles: “(S)” denotes the Pull Request Submitter
role, “(R)” the Pull Request Reviewer.
(S) Explore list of issues on repository, choose issue to fix, make
note of issue number.
(S) Edit a file to fix the issue, commit changes to a new branch
with an appropriate name. Ensure that the changes include some typo etc
that the Reviewer will be able to comment on and/or suggest a
fix/improvement to.
(S) Show how to open a pull request from that new branch to
main.
(S) In Pull Request description, reference the number of the issue
being fixed, using a keyword “Fixes #X” or “Resolves #X” (where
X is the issue number) to ensure automatic closure.
(S) Demonstrate how to mark the PR as a draft, and then how to mark
as it as ready for review.
(S) Request a review from the Pull Request Reviewer.
(R) Open the new Pull Request in GitHub.
(R) Explore the diff of the PR, by opening the Files
changed tab and talking through the display.
(R) Comment on a line in the diff.
(R) Suggest a change to the diff, using the “Add suggestion” button
or three backticks marked suggestion.
(R) Complete your review by adding some summary text and selecting
“Request changes”.
(R) Return to Conversation tab, to show how the PR thread
looks after the review has been submitted.
(S) Navigate to PR and the Files changed tab.
(S) Respond to Reviewer’s comment(s), accept their suggested changes
(draw attention to the option to batch multiple suggestions into a
single commit).
(S) Return to the Conversation tab and request a new review
from the Reviewer.
(R) Look again at the PR diff (Files changed) and ensure
that requested changes have been made.
(R) Approve the PR.
(R) Thank the Submitter for their contribution.
(R) Merge the PR.
(R) Navigate to issue listing to confirm that the relevant issue has
been automatically closed.
Practice with Pull Requests (15 minutes)
In breakout groups, assign yourselves the role of Submitter (S) and
Reviewer (R), and repeat the process described above by completing the
steps below. If there are any issues open on the repository that can be
solved very quickly, feel free to address those with this exercise.
Otherwise, the Submitter can open a Pull Request to make simple changes:
typo fixes, or some completely arbitrary changes to a file. If the
changes made for the exercise are not improvements to the lesson, the
Reviewer should follow all the steps but close the PR at the end instead
of merging it.
(S) Edit a file and commit the changes to a new branch.
(S) Open a new Pull Request from this new branch to
main. If your PR will fix an open issue on your repository,
include “Fixes #X” at the start of the PR description, where X
is the issue number.
(S) Request a review from the Reviewer.
(R) Review the PR, making comments and suggesting changes as
appropriate. When you have finished, approve the PR or request changes,
based on your review.
(S) Respond to any comments from the Reviewer and/or, if any changes
were requested, make those and re-request a review when you have
finished.
(R) When all changes have been made to your satisfaction, approve
the PR, thank the Submitter for their contribution, and merge the PR (Or
close it if the changes were made for practice but should not be
included in the actual lesson.)
Repository administrators can enforce this workflow by
protecting the default branch of the GitHub repository:
requiring anyone without Admin access to submit changes by pull
request instead of committing changes directly to main.
Making effective review comments in a PR is something you will get
better at with practice - here is some general guidance to help with
reviewing a PR from a collaborator:
Good reviews have empathy - consider the contributor’s feelings,
identify positives and try not to be judgemental.
Aim for improved/better rather than perfect.
Make sure your review comments are specific and actionable.
Make it clear in the comment what you want changed as part of a PR
and, ideally, provide a concrete suggestion.
Only request changes if the content is wrong or hard to understand
(remember, different does not always mean better).
If a conversation on a PR hasn’t been resolved by one or two
back-and-forth exchanges, consider scheduling a meeting to discuss
further.
If you need more help or practice with GitHub features, you can
attend a GitHub skill-up session for Carpentries Maintainers and lesson
developers which uses the following
curriculum. These sessions will be periodically announced on The
Carpentries calendar of events, The Carpentries Slack and
mailing lists.
Managing Communication
Having an open, publicly-visible list of all the issues with your
project is a helpful way of letting people know you are aware of issues
and you are working on them. This can indicate to an external audience
that the project is active. It also provides you and your collaborators
with an “at a glance” view of the state of the project, making it easier
to prioritise future work.
GitHub provides a useful notification feature for collaborative work
- mentions. The mention system notifies team members when
somebody else references them in an issue, comment or pull request - you
can use this to notify people when you want to check a detail with them,
or let them know something has been fixed or changed (much easier than
writing out all the same information again in an email!). You can use
the mention system to notify individual GitHub accounts or whole teams
(when you need to mention multiple people at once and it is not
practical to list them all individually). Typing @ in
GitHub will bring up a list of all accounts and teams linked to the
repository that can be “mentioned”. You can also mention people who
aren’t linked with the repository but they won’t be listed in the
@ drop-down list so you will need to know their GitHub
username in order to mention them.
Slack is commonly used in the
Carpentries community for quick, day-to-day message exchange among
community members and staff. You should join the Carpentries Slack
organisation in particular to connect with other lesson developers.
Once you join, you can create a channel for your lesson development
project under the Carpentries
public Slack workspace or if you need more complex team
communications you can create your own Slack workspace for free. Note
that the Carpentries Slack is an enterprise workspace so all messages
and files will be retained and no messages will be lost (for free
workspaces only the most recent 10,000 messages can be viewed and
searched and file storage limit is 5 GB).
Collaborators can also use other platforms to discuss lesson
development or receive contributions from newcomers who are not yet
fluent in using GitHub’s systems of communication. The Carpentries can
assist with creating a mailing list specific to the development of your
lesson on their TopicBox
platform for managing threaded email discussions. Also make sure to join
the Incubator
lesson developers mailing list on TopicBox to keep an eye on
announcements and discussions relating to lesson development in general
within the Carpentries community.
Recommended Reading
Here are some additional resources where you can learn more about
collaborating with your team in an Open Source project:
GitHub’s features Issues, Pull
Requests and Mentions can all be used to
transparently and effectively communicate and collaborate in a team
working on a lesson development project.
Slack and emailing lists provide communication channels outside of
GitHub - for quick day-to-day messaging and more permanent discussions
around lesson issues and development, respectively.
What information should I provide about the project to potential
collaborators?
How can I ensure regular progress on lesson development?
Objectives
After completing this episode, participants should be able to…
Adjust a lesson repository to attract potential new
collaborators.
Create and modify issue and pull request templates in GitHub.
Identify strategies to encourage regular development and maintenance
of a lesson.
You may start working on projects by yourself or with a small group
of collaborators you already know, but open source projects in the
Carpentries community often grow beyond that small group. Community
collaboration is the cornerstone of The Carpentries - all curricula and
various programmes are developed together by the community members
across borders, domains and backgrounds. Most initiatives are done
through community consultation and collaboration is used as a pathway to
empower people, give members a sense of purpose in the community and
realise shared goals.
Collaboration helps bring different perspectives to solving a problem
and share the load of the required work to develop, pilot, and, later
on, maintain the lesson. People drop in and out of projects for various
reasons and their levels of involvement may vary over time due to other
commitments - for this reason it is worth taking time from the outset to
make a project welcoming for newcomers in order to achieve a steady
number of people working on your project at any time. You should plan to
make it easy for new collaborators to join, to set up a local workspace
so that they can contribute, help them find tasks so that they know what
to contribute, and make the contribution process clear so that they know
how to contribute. You also want to make it easy to give people credit
for their work on your project as well as for others to reuse and give
credit to your project. In this episode, we cover best practices for
collaborating with newcomers and tools available to help. And remember,
anything you do to make it easier for newcomers to contribute will be universally
useful for all collaborators.
Documenting Your Lesson
For collaborative projects, it is important to spend the time on
‘external-facing’ features, such as documentation, to make the project
as welcoming and as easy to get involved in as possible for newcomers
who may not have a deep grasp of the project or know all its existing
problems. Such documentation will be useful to yourself and other team
members as well, e.g. if you are trying to come back to the project
after a break or are reusing it for a new collaboration in the
future.
Your lesson documentation should contain the following information,
which should be kept up to date.
Description of the Project - README File
It is helpful to share an example of a README when teaching this
section. Use your own example if you have one, or otherwise share the
README from the source repository of this training curriculum. We
try to keep it updated and aligned with the recommendations here. Please
let the Maintainers know by opening
an issue or a
pull request if you identify anything that could be improved.
README, README.txt or
README.md is a plain text or Markdown file located in the
project’s root directory representing the default landing or home page
for repositories in GitHub (and similar project repositories) by
convention. It represents the first piece of documentation that people
will see when visiting your lesson repository, hence it should concisely
explain what the lesson is about and who it is for, and contain links
and pointers to further information. For example, README
should include:
lesson title
lesson description
rendered version of the lesson - even though this
is typically included in the About section of the
repository, it is always useful to link to the URL where the rendered
lesson is available
contact information - include up-to-date email
addresses or mailing lists or other details on how to get in touch with
the lesson maintainers
contributing information - this is an opportunity
to list what kinds of contributions are sought (and what are not) and
how to get involved in lesson development for new contributors. You can
provide more details in a separate CONTRIBUTING file within
the repository’s root directory and link to it from
README
credits/acknowledgements - make sure to credit
those who have helped in the lesson’s development or inspired it, and/or
any resources/templates that you have reused. You can also link to a
separate AUTHORS.md file within the repository’s root
directory to list all the people who contributed to the lesson content
(if this list starts to become too large to include in the
README itself)
citation - a convention is to include the citation
information for your lesson in a separate CITATION file
within the repository’s root directory and link to it from
README.md so others can cite the use of the lesson in their
own publications and media and it can also be automatically discovered
by other applications. These CITATION files are discussed
in more detail below.
license - a short description of and a link to the
lesson’s license typically contained in a separate LICENSE,
LICENSE.txt or LICENSE.md file within the
repository’s root directory. The lesson repository created from the
Carpentries lesson template already contains a default
LICENSE.md file, but you should modify this to more
accurately describe how the lesson content can be re-used by
others.
Helping People Cite Your Lesson
The default citation file for new lessons using The Carpentries
Workbench is CITATION.cff, in Citation
File Format (CFF). CFF is defined using YAML (which we already
encountered in the config.yaml file for the lesson website)
and is the
recommended way of storing citations in GitHub. If your lesson
repository contains a CITATION.cff file, GitHub will
automatically show the citation information in the sidebar, making it
more visible and accessible for visitors to your repository. CFF is also
recognised and supported by other platforms including Zenodo and the Zotero reference manager.
The CITATION.cff file for newly-created lessons contains
only placeholder information, which should be replaced with relevant
details for your project as soon as practical. Citation information can
also be contained in a plain text file (CITATION,
CITATION.txt) or a Markdown file
(CITATION.md). In all cases, it is up to the lesson
developers to decide what information to include in their citation
file.
Digital Object Identifiers
We recommend that you obtain a Digital
Object Identifier (DOI) for your lesson as soon as feels
appropriate, at the latest when the lesson reaches the beta phase. You
can use this DOI in the citation information (e.g. in
the identifiers field of your
CITATION.cff), to allow people to cite a particular
version of your lesson, e.g. a snapshot of the lesson when it entered
beta testing, captured as a Zenodo record.
Instructions for Contributors - CONTRIBUTING File
CONTRIBUTING, CONTRIBUTING.txt or
CONTRIBUTING.md is a text or Markdown file within the
repository’s root directory where you should provide detailed
description of the ways people can send their contributions, what kinds
of contribution will be credited and in what ways. For example, at what
point would someone be listed as an author and how you will credit
contributions that are not recorded in the commit history of the
project. The latter is particularly important for newcomers who may not
be proficient with the use of GitHub’s features for collaborative work,
such as issues, mentions, pull requests and code review, but could still
provide valuable input and contributions.
Carpentries lesson repositories already have a generic Carpentries
CONTRIBUTING.md file to get you started. You should review
it and, if need be, modify it to fit your team’s needs and way of
working. For example, you may want to expand on what contributions you
are not looking for if your lesson already contains more
material than can be covered in a typical workshop. You can also detail
all channels on which you are willing to receive contributions on, if
they are different from the defaults included in the file.
Issue and Pull Request Templates
Consider setting up issue
and pull request templates to help newcomers who may not have much
experience working on collaborative projects in GitHub. Such templates
can provide a structure for the issue/pull request description, and/or
prompt them to fill in answers to pre-set questions. Both can help
contributors raise issues or submit pull requests in a way that is
clear, helpful and provides enough information for maintainers to act
upon (without going back and forth to extract it). GitHub provides a
range of default templates, but you can also write
your own.
Other Documentation
Once you have set up the basic documentation about your lesson, you
may consider adding the following useful information to your
documentation too:
set-up guides for installing and rendering the lesson locally. In
most cases, a link to The Carpentries
Workbench documentation will be sufficient. You can also contribute
to the Workbench documentation to help improve it for the
community.
how to create and modify the pages in the lesson
Templates for collaboration
We have prepared the following templates to help you get started with
writing the README and contributing guide for your project:
Exercise: preparing your repository for collaboration (15 minutes)
Spend some time doing one of the following:
Modify the README.md and CONTRIBUTING.md
files in your repository to provide the relevant information for
would-be contributors to your lesson:
the name(s) of the current developer(s), linked to their GitHub
profile(s) (README)
contact information (README, CONTRIBUTING)
guidance on how people should contribute, and what kinds of
contribution you are most interested in receiving (CONTRIBUTING)
links to any important background information (README)
links to any funding bodies or host institutes that are supporting
the development of the lesson (README)
Create a new issue or pull request template, or modify an existing
one, to guide contributors on how best to begin collaborating with you
on GitHub.
Using the
cffinit webtool, create a CITATION.cff
with information appropriate to your project (listing authors, including
the lesson title, repository URL, etc) and overwrite the current
placeholder content with the result.
Groups of collaborators taking this training together should discuss
first how they will assign these tasks between them.
Collaborating on a Lesson
Boosting the Visibility and Attracting New Collaborators
In addition to having the complete documentation in the lesson
repository, The Carpentries community provides a number of ways to
further raise the visibility of the lesson among the broader community
and encourage community members to contribute to its further
development. For example:
writing a blog post about the lesson for The Carpentries Blog, and/or
attending a community discussion call to promote the lesson
advertising the lesson at various Carpentries mailing lists -
e.g. general
discussion, instructors,
regional communities or specific curriculum lists
You should also consider organising and advertising “lesson
sprint”/“collaboration drive” events where you can help newcomers get
started as contributors to your project, and/or including such a session
in the programme of existing events (e.g. as an interactive session at a
conference/community meeting). The Carpentries community handbook
includes a set of recommendations
for organising and running a lesson sprint.
Issue Labelling for Newcomers
You can encourage contributions to your lesson from newcomers by
using specific labels on issues to highlight suitable opportunities to
help. Apply the good first issue or
help wanted labels to issues in your repository to indicate
that the maintainers will particularly welcome help or pull requests
fixing such issues.
Noticing When Something Happens
In addition to Mentions, GitHub implements a
comprehensive notifications
system to keep you up-to-date with activities in the lesson
repository so you can react in a timely manner. Check out GitHub’s
documentation on setting notifications on individual repository. You
can choose whether to watch or unwatch all events on an individual
repository, or can choose to only be notified of certain event types
such as issues, pull requests, mentions, etc.
The next spoiler contains a lot of information, and can require a lot
of time to discuss fully while training. We recommend that you do not
dwell on these details while teaching unless you have time to spare.
Instead, you could share a link to the
spoiler and mention to trainees that this information can be helpful
if they are struggling with GitHub notifications.
If you work on multiple projects, or the projects you follow on
GitHub are particularly active, the volume of notifications you receive
can quickly become overwhelming. Here are some approaches you can take
to help you stay on top of things, and distinguish the high-priority
tasks and important updates from the regular traffic.
Email Notifications
If you want to filter, organise, and redirect email notifications
from GitHub, here are some characteristics of the messages that you make
use of:
All GitHub notifications are sent from the address
notifications@github.com.
Notification emails are sent to the address
repo-name@noreply.github.com, where repo-name
is the name of the repository where the notification was triggered.
The email subject begins with [org-or-user/repo-name],
where org-or-user is the name of the organisation or the
username of the user who owns the repository and repo-name
is the name of the repository where the notification was triggered.
In addition to your email address, the cc field of the message
contains an address that describes the type of event that triggered the
notification, e.g. author@noreply.github.com for activity
on an issue or pull request that you opened,
mention@noreply.github.com for a mention of your username,
or team-mention@noreply.github.com for a mention of a team
you are a member of, etc.
The email header (metadata) includes a mailing-list
field with an identifier in the form
repo-name.org-or-user.github.com, which can be used to
filter by the project and/or its owner.
Most email clients provide configuration for rules that can be set to
redirect messages to particular folders, and/or to annotate them with a
mark or flag, based on this kind of information. Here is the
documentation for setting such rules in Gmail.
Notifications on GitHub.com
The alternative to using email to keep track of project activity is
to manage notifications on GitHub. Adjust how you receive notifications
in the
Notifications section of your account settings. When logged
in, you can visit https://github.com/notifications to see notifications
for your account, presented as a table. Through this interface, you
can:
group notifications by repository.
see notifications for a particular repository.
see particular types of notification e.g. mentions, issue
assignments, etc.
create more sophisticated filters using the search bar, e.g. all
mentions for all repositories in a particular organisation.
The account notification settings allow you to specify certain types
of notification that should be sent by email, while everything else is
collected in the web interface. Whatever strategy you choose, one of the
most important habits that will help you stay on top of your projects
and tasks is to remember to check these notifications frequently.
Notifications for Your Lesson Repository
As a collaborator with edit access to the repository, your
notification settings on the lesson repository should already be set to
All Activity.
You can review the level of communication you will be receiving about
activities in the repository by clicking the ‘Unwatch’ button on your
repository homepage: the dropdown menu will show you a range of options
for the different kinds of events that you will be notified about on the
repository. ‘All Activity’ is a great choice to begin with because it
will ensure that you do not miss out on any changes/events.
As the time progresses, feel free to revisit this setting to adjust
the level of notifications to the right level for you.
GitHub’s different notification settings when
watching repository events with ‘All Activity’ highlighted
Saying “No”
Not every contribution is a good fit for a project and, especially as
your lesson becomes more stable, you probably will not want to accept
every suggested change. If you receive a pull request or issue that does
not fit to your lesson, consider the following points to help you
politely decline without demotivating the contributor from contributing
to your lesson (or another open source project) again:
Thank them for taking the time to contribute.
Explain why the contribution does not fit into the
lesson, and offer suggestions for improvement if you’re able to.
Link to relevant documentation, if you have it.
Notes about the design of the lesson (e.g. from this training) and any
relevant discussion threads can be very helpful here. If you notice
similar repeated requests/contributions, you might want to address them
in your documentation (e.g. in the CONTRIBUTING.md file) to
save yourself time in future.
The following practices have been shown to help maintain steady
progress with lesson development:
being responsive to notifications about activities and mentions
scheduling regular co-working/sprinting sessions with team members
(e.g attaching your sprint sessions to other open source community
activities, which may offer goodies, rewards and prizes for
participants, can provide motivation and activity spikes)
working alongside other members of The Carpentries community at
Maintainer or lesson development co-working sessions
blocking time in your calendar for issue triage/solo material
writing
planning lesson pilots in advance to help set targets
Make sure to join the Incubator
lesson developers mailing list to keep an eye on announcements and
discussions relating to lesson development in the Carpentry
community.
Exercise: planning to collaborate (10 minutes)
Use this time to plan how you will continue to make progress with
developing your lesson, and ensure that you are ready to respond to
external contributors when they arrive at your lesson. This time is
yours to do with as you think is best for yourself and your project, but
you might consider using it to do one or more of the following:
schedule a lesson development co-working session with your
collaborators, and/or mark the next community co-working session in your
calendar.
ensure that your the notification settings are appropriately
configured for your lesson repository.
contact The Carpentries team to ensure issues from your lesson will
be included on the Help Wanted page.
Recommended Reading
Here are some additional resources where you can learn more about
encouraging newcomers to get involved with your Open Source project:
You should make an active effort to attract potential collaborators
and try to make them all feel welcome and included. The Carpentries Help
Wanted page and featuring in the
Incubator Lesson Spotlight can boost the visibility of your lesson.
Creating the appropriate documentation and using GitHub features such as
labels, and issue/pull request templates will help lower the barriers
for contributions to your project.
When you design your lesson with new contributors and increased
accessibility in mind, you make things better for everyone in the
process.
Scheduling regular co-working sessions, blocking time in the
calendar for issue triage, and setting and being responsive to GitHub
notifications will ensure regular progress on the lesson.
How can I use GitHub to manage a lesson development project?
What strategies exist to help collaborators make decisions and
govern an open source project?
Objectives
After completing this episode, participants should be able to…
Use GitHub features to help you with planning and managing your
project.
Prepare for a decision-making meeting between collaborators.
Project planning & management
Developing a lesson is a project and, like most projects, it consists
of multiple tasks. Keeping track of the list of tasks the team has to
do, progress on each, prioritising tasks for future development, sprints
and releases, etc., quickly becomes a non-trivial task in itself.
Without a project management tool/framework, it can be hard to keep
track of what’s done, or what needs doing, and particularly difficult to
convey that to others in the team or share the responsibilities.
Different tools and platforms exist to help you with project planning
and management - for example, GitHub’s Milestones
and Projects,
Asana, Trello, Miro, even sticky notes on a white board.
Different tools will work best for different teams of collaborators, but
making sure you use any tool to plan and manage work on your lesson
development project is better than making it up as you go along.
Project Management in GitHub
Carpentries lessons are developed in GitHub so it is convenient (but
not a must) to use GitHub’s Milestones and Projects to help you keep
track of different components of the project and their current
status.
Milestones are a way of organising issues on your project into
smaller units of work (e.g. deliverables) that have deadlines and
progress of which needs to be closely tracked (e.g. release management).
Projects provide a way of visualising and organising the work on a
higher level and are more suitable for project management tasks that are
more continuous and not time-bound. While a Milestone is associated with
a single repository, a Project can span over multiple Milestones and
even different repositories within an organisation in GitHub.
Milestones are good for managing time-bound, structured iterations
and working in short sprints on batches of issues. Dedicated sprints
seem to work well for lesson development, so we will have a look at how
Milestones can help with this. We recommend you to also have a look at
GitHub’s
documentation on Projects and see if they are suitable for your
lesson development workflow.
GitHub’s Milestones
A Milestone is a collection of issues to be worked on in a given time
period. Milestones are good for tracking progress as they provide an
ordered list of issues which must all be resolved for the overall task
to be considered complete. Milestones work like labels - they mark and
group issues that are expected to be closed at some point in time.
The Milestones page in GitHub is accessible from the
Issues or Pull requests pages - from there you
can create a new Milestone, set its description and due date, sort all
Milestones by due date and see the percentage of issues completed for
your Milestones. Within an individual Milestone’s page, you can see the
list of associated issues and prioritise them by pointing to the left of
an issue until the ‘hand’ pointer appears and then drag-and-dropping it
to a new location in the list (with higher position in the list
indicating higher priority). Back on the
Issues/Pull requests page, you can assign
issues/pull requests to a Milestone and filter your issues/pull requests
by Milestones.
Community members’ perspectives on project management
Sven van der Burg, Research Software Engineer at the Netherlands
eScience Center, and Carpentries Instructor and Lesson Developer,
describes his group’s approach to piloting and contributing back to
someone else’s lesson in The Carpentries Incubator:
We are not direct maintainers of the lesson, but these issues are
important for us to fix before teaching the next edition of the
workshop, because we don’t want to run into them again. Of course other
users of the lesson would also benefit from the issues being
fixed.
Therefore, we organised a one-day sprint in which we gathered
together colleagues from different teams to work on improving the
lesson. By all working together on one day, together in the same room,
we were able to focus and get a lot done in a short amount of time,
while not being distracted by the hundreds of other things we are all
working on. Of course, we contacted the lesson maintainers to make sure
that our efforts would be welcome by opening an
issue in their lesson repository.
We used a Microsoft Planner board to organise our work and keep
track of what needs to happen:
In this way, we could adapt the lesson to our own needs and
contribute to the lesson in The Carpentries Incubator in a smooth way
that does not take away focus on other projects of me and my
colleagues.”
Renata Curty, Research Facilitator for Social Sciences at UCSB,
describes her group’s approach:
Our group uses issues and labels and sets milestones in GitHub to
organize our lesson development work.We also use a spreadsheet
to track higher-level decisions, tasks, and assignees and record ideas
and discussion points in our meeting notes before converting them into
more detailed issues.We have been meeting regularly (weekly or
biweekly) to maintain momentum, which helps us to make small but steady
progress.We are currently working on a modularized version of
the workshop that will allow instructors to choose what to teach based
on the time, cohort needs, and learners’ entry knowledge.
Exercise: prioritising issues & planning work (15 minutes)
Use Milestones to prioritise issues and plan the first sprint and/or
the first lesson release with your team.
Depending on how many issues were registered on your repository, it
is possible that you will not be able to work on all the issues in the
first sprint. As the development of your lesson progresses, any
remaining unresolved issues can be reconsidered and prioritised for
another sprint/milestone.
This process of receiving new issues and requirements,
prioritisation, and working on them is naturally continuous - with the
benefit that at key stages your team is repeatedly re-evaluating what is
important and needs to be worked on which helps to ensure real concrete
progress against project goals and requirements which may change over
time.
Project Governance
In addition to managing your project on a day-to-day basis, you
should consider a governance model for your lesson to describe the
ground rules of participation, the roles that project participants can
take on and the process for decision making within the project. You may
wonder why you’d ever need a formal governance process - after all,
everyone in your team is collegial, polite and hard-working. However,
even among the most harmonious teams you will likely realise you need an
agreed way to make decisions and resolve conflicts when your team
encounters their first real disagreement1.
The moment your collaborator group exceeds 3 or 4 members (and fewer
if the collaborators do not all know each other equally well at the
outset) you should establish and document some kind of governance
process. It will help mitigate any power imbalances which are expressed
when one team member (or a group of team members) are more vocal and are
able to dominate the decision-making process. It is advised to make
these considerations early on, before your project grows too much and
introducing structure and process into it becomes more difficult and
complicated. Besides, if you do not establish these processes from the
start, you are likely to discover the need to do so only when you first
encounter disagreement within the team: a situation unlikely to provide
perfect conditions for a discussion of governance.
GitHub’s Minimum Viable
Governance (MVG) for establishing a lightweight governance process
into free and open source projects that are run in version control
systems.
In MVG decisions are made through consensus of the project
maintainers or determined based on the maintainers’ consideration of a
number of factors in “good faith” (when explicit agreement of all
maintainers cannot be reached). In reality, consensus-based governance
can run you into trouble when you hit an issue where maintainers do not
all agree but still want to make progress. Also, the definition of “good
faith” is open to interpretation and leaves plenty of room for people,
especially the powerful, to get away with acting in bad faith.
A system based on voting and majorities may help you get around this
issue. MVG is designed for a one organisation-many projects setup and is
too elaborate for a single lesson project. However, it includes
“boilerplate” template files for a steering committee, charter,
governance, Code of Conduct documents, and language laying out a
preference for consensus but with voting as a fallback, and other
templates that may be helpful for those starting out and unsure about
how to get on with establishing a governance process.
These activities are intended to help you think about the human
relationships between collaborators on a project and how your
interactions can influence the progress you make.
Discuss with your colleagues which, if any, aspects of this would
work for your team and which would not. What model of governance and
decision-making might work for your team?
If you have time remaining, discuss what factors influence how much
power each member of your team has in the context of the project. This
could be “hard”/formalised power, e.g. project collaborators in more or
less senior positions within the same workplace, and/or “soft”/informal,
e.g. differences in the amount of time each collaborator has available
for the project, or different levels of expertise in the subject matter
of the lesson, or the lesson infrastructure and associated tools such as
Git and GitHub.
Write down any questions you may have for your trainers.
(Recommended reading: the Dealing with Disruptive
Behaviours resource linked in the exercise for lesson
developers working alone, below, may also be helpful to ensure smoother
project progress in your team.)
The Dealing with Disruptive
Behaviours resource from the US National Oceanic and
Atmospheric Administration (NOAA) Office for Coastal Management provides
guidance for meeting facilitators. It includes examples of several
common behaviours that can be disruptive to meetings (and therefore to
efficient progress on a project) and advice for facilitators on how to
intervene and mitigate them.
Open the Dealing with Disruptive Behaviours resource,
select each behaviour, and read the descriptions.
Which of these behaviours do you recognise in yourself? Put another
way, which “character” (“Talkative Blowfish”, “Apathetic Flounder”, etc)
do you most resemble?
Think about another project you work on where you are part of a
team. Can you identify any of the disruptive behaviours in your
colleagues? If so, can you find anything in the suggested interventions
for these behaviours that could help you in your shared project?
Recommended Reading
Here are some additional resources to learn more about Open Source
project management/governance:
Being intentional about task/project management for your lesson
development project is better than making it up as you go along.
Different tools and platforms exist to help you with project
management - chose one that suits your team best.
Dedicated work sprints seem to work well for lesson development and
GitHub’s Milestones can be used effectively to plan and
manage such sprints.
Make sure to consider and establish a governance model for your
project early on - to describe the ground rules of participation and the
process for decision making within the project.
What steps can you take to be a good collaborator?
How can we improve this training?
Objectives
After completing this episode, participants should be able to…
Draw a diagram or compose a written summary of the key concepts from
this training.
Articulate constructive feedback.
We have reached the end of this training.
Looking Back
Throughout this training we promoted a collaborative approach to
lesson development (as many other initiatives in The Carpentries
community are conducted). Collaboration helps bring different skills and
perspectives to a project and share the load of the work required to
bring the project successfully past the finishing line. For lesson
projects, this shared effort also extends to maintaining and updating
the lesson after the initial development phase.
Exercise: What steps can you take to be a good collaborator? (20 minutes)
Spend some time reflecting on collaboration aspects we covered in
this training and exploring at least two of the following lesson
repositories, noting down what things you noticed in these repositories
that could aid collaboration:
Trainers will then lead a discussion on what people found and steps
you could take to improve your lesson repository.
Collaboration Top Tips
While you will know the best collaboration practices for your team,
here are some suggestions:
Provide ‘external-facing’ features for your lesson (such as
description, objectives, contribution and ‘how to get involved’ guides,
preferred way of communication) - these will be universally useful for
everyone coming to your repository (your team included).
Find the best tools that work for your team for communication,
documentation and knowledge sharing, issue & project
management.
Document and share knowledge within the team to facilitate team
collaboration. Share decisions and communicate with your team
synchronously and asynchronously.
Plan your meetings to get the most out of your team (e.g. set
agenda, assign meeting roles - chair, note-taker, time-keeper, etc.) and
be appreciative of people’s times.
Record and manage issues with your lesson as you develop, trial and
maintain it (e.g. assign people responsible for issues, categorise and
prioritise them).
Assign tasks and roles to distribute the responsibilities - this
includes assigning larger roles (project management & governance) to
team members and assigning specific issues/tasks to collaborators.
Progress is much more likely to be made when people know what they and
others should be working on.
Dedicated sprints where the whole team works on a set of issues
towards a milestone work well for lesson development and making progress
on projects in general.
Be open and welcoming to new contributors and provide different
opportunities to get involved with your project (from novice to expert
collaborators). Point to upskilling guides (or run upskilling sessions
if possible) to bring people up to speed.
Manage contributions in a timely manner by noticing when things
happen and responding to new issues, comments and Pull Requests (PRs).
Show appreciation for everyone’s contributions.
People bring different perspectives, expertise and skills to a
collaborative project and, while they can be productive and harmoniously
complement each other, having some decision making and conflict
resolution process in place will make sure progress can be made when
disagreements arise.
Looking Ahead
By now, you should have made good progress towards a first draft of
your lesson.
Referring back to the life cycle stages for
lessons described earlier in the curriculum, your lesson is still in
pre-alpha but hopefully approaching alpha, when it is
ready to be taught in full for the first time.
As you and your collaborators continue to iterate over the lesson -
teaching it, gathering feedback, reflecting on the experience, and
improving it for next time - the lesson will pass through alpha
into the beta and stable stages (perhaps via peer review in The Carpentries
Lab).
Your role as the original developers of the lesson, and the kind of
interactions/contributions taking place on your repository, will change
as you move through these stages.
Life cycle stage
Type of contribution
Your role
pre-alpha
new content and whole new episodes; changes/additions to lesson
objectives
active development; onboarding new collaborators; project
management
active development; reviewing changes; documenting feedback/actions
based on pilot workshops; project management
beta
feedback from pilot workshops; polishing changes; expanding
instructor notes; suggestions for major overhauls/reorganisations
recruiting pilot instructors; reviewing changes; guiding
contributors
stable
feedback from workshops; polishing changes; new/updated content
reviewing changes; guiding contributors; onboarding new maintainers;
stepping away from the project
As you continue to make progress, remember that The
Carpentries Lesson Developer Handbook provides guidance and a list
of resources for lesson developers to consult.
Farewell and Feedback
Thank you for following this training. We hope you found it helpful
to build a foundation for your current lesson project and to prepare you
to collaborate on other projects in the future.
To bring the training to a close, the Trainers will lead a number of
activities to gather feedback and help you reflect on the
experience.
One Up, One Down (10 min)
Provide one up, one down feedback on the entire Collaborative Lesson
Development Training. Remember:
Say only one thing, and try not to duplicate. This gets harder for
those who come later!
Trainers should try not to respond, only record responses (e.g. in
the Etherpad). This is also hard, but important!
Reflection exercise (15 min)
Take this time to reflect on what you have learned in this last part
of the training, and take some action to plan to successfully complete
the development of your lesson after you leave. It is up to you how you
use this time, but you might find some inspiration in the prompts
below.
How could you use the project management features of GitHub to
prepare for the next stage of the lesson development process?
If you have been working in a collaborative group, what steps could
your group take to improve the way you work together for the remainder
of the project?
If you have been working on the lesson alone so far, what steps
could you take to find new collaborators to help you with the further
development of your lesson?
Based on the experience of teaching a section of your lesson, do the
objectives you wrote for your lesson still seem realistic? Can you
identify anything that can be added or removed now that you have
completed a trial run?
Key Points
Documenting your lesson project, curating your lesson repository,
welcoming new contributions, and taking advantage of GitHub’s project
management features all make it easier for people to collaborate with
you.
Any lesson can be improved with feedback, including this one.
