Content from Introduction
Last updated on 2024-02-02 | Edit this page
Overview
Questions
- What is covered in this training?
- Who are the trainers?
- Who is participating?
Objectives
After completing this episode, participants should be able to…
- Explain how trainers and participants will interact throughout the workshop.
- Summarise the main skills that will be taught in this workshop.
Note: some of the content in this course is reproduced or adapted from The Carpentries Instructor Training curriculum.
Pronouns and Names
Using correct names and pronouns (e.g. “she/her”) is important to setting a tone of respect. Learning these is hard to do quickly, so we recommend displaying it prominently during the workshop.
In an online workshop, give everyone a moment to update their display name to reflect how they would like to be addressed.
At an in-person event, we recommend supplying name tags and markers, or using plain paper to create table-displayed name placards.
Note that pronouns are personal and some participants might prefer not to share them. Do not force people to share their pronouns.
Before The Training Begins
Getting to Know Each Other
If the Trainer has chosen an icebreaker question, participate by writing your answers in the shared document for the workshop.
Code of Conduct
To make clear what is expected, everyone participating in The Carpentries activities is required to abide by our Code of Conduct. Any form of behaviour to exclude, intimidate, or cause discomfort is a violation of the Code of Conduct. In order to foster a positive and professional learning environment we encourage you to:
- Use welcoming and inclusive language
- Be respectful of different viewpoints and experiences
- Gracefully accept constructive criticism
- Focus on what is best for the community
- Show courtesy and respect towards other community members
If you believe someone is violating the Code of Conduct, we ask that you report it to The Carpentries Code of Conduct Committee by completing this form.
Today’s Trainers
To begin class, each Trainer should give a brief introduction of themselves.
(For some guidelines on introducing yourself, see the Workshop Introductions section of the Instructor Training curriculum).
Now, we would like to get to know all of you.
Our First Exercise (10 minutes)
What was the best lesson you ever followed (were taught in a class, read through online, read in a book)? Try to differentiate between what was good about the performance of the teacher/trainer and what was good about the content of the lesson itself. Take a few minutes to write down some notes about your answer, then introduce yourself to the other participants and tell them about it.
Collaborative Lesson Development Training Overview
The main objective of this training is to teach you the skills you need to design and develop an effective lesson, in collaboration with other members of the community.
During the training, we will introduce the steps you can take to design and develop a lesson to meet the needs of your target audience, and give you time to begin implementing those steps during the workshop itself. By the end of the training, you can expect to have defined an outline for your whole lesson, and begun filling in a detailed plan for some of its individual sections. The content you create in the training will exist as an open source lesson website similar to the one this training is based on. You should also know how you can continue building on the lesson and some plans for how you will collaborate on the project after the training ends.
We will focus on three main areas:
Designing a Lesson
Much of the training will discuss a process to incorporate good practices in lesson design. We will explore how defining the specific skills you wish to teach early on in the development process provides a foundation from which you can build a stronger, more impactful lesson.
Building a Lesson Website
Throughout the training, while you design and begin developing the content of your lessons, we will teach you how to incorporate this into an organised and accessible website using our lesson infrastructure.
Collaborating Effectively
We believe that lessons are much more likely to succeed, and to remain useful in the long term, if they are developed collaboratively. This training will discuss some of the ways that you can welcome new collaborators and work effectively and efficiently with those you already have.
Learning How to Teach a Lesson
This training will focus on the content - how to prepare a good lesson. More about the performance - how to deliver a lesson most effectively - is covered in The Carpentries Instructor Training.
Key Points
- This training aims to teach you a process for designing a lesson and the skills to develop it as an open source website, in collaboration with others.
Content from Lesson Design
Last updated on 2024-10-17 | Edit this page
Overview
Questions
- What are the recommended steps to take when developing a new lesson?
- What lesson do you want to develop during and after this workshop?
Objectives
After completing this episode, participants should be able to…
- Explain the lesson design process we will be adopting for this course.
- Summarise the lessons that participants will be working on.
A Lesson Design Process
In order to design an effective lesson, we need a structured approach with the learner in mind and clearly identified goals. Throughout this training, we will use a modified version of Nicholls’ five phase paradigm for curriculum design1. Nicholls’ paradigm describes a process, commonly referred to as backward design, where those who wish to develop a new curriculum first begin by defining exactly what their learners will be able to do after they have completed the lesson/training/course. The subsequent stages of the curriculum design process involve designing content to directly meet those stated outcomes.
- Select learning outcomes
- Choose learning experiences to help learners achieve these outcomes
- Develop content to support these experiences
- Assess learner progress towards desired outcomes
- Evaluate chosen outcomes, experiences, and content based on this assessment
The last two phases of Nicholls’ paradigm involve assessing learner progress towards the desired learning outcomes and evaluating the stated objectives and current content in light of the results of that assessment. In The Carpentries, most workshops are relatively short-format, without room for an extensive assessment after the teaching has finished (a summative assessment). To account for this, our lessons place an emphasis on formative assessment: assessment of learner progress that takes place while the teaching is still going on, to give instructors opportunities to evaluate the teaching and lesson content before the end of the workshop.
To account for this, we have adapted Nicholls’ five phases in this training, to place an emphasis on assessing learning during a workshop:
- Define desired learning outcomes
- Design assessments to determine progress towards desired outcomes
- Write content to lead learners from one of these assessments to the next
- Assess learner progress towards outcomes during teaching
- (After the break) evaluate how closely the outcomes meet the objectives
Note the cyclical nature of this process: you will complete one iteration through this cycle during this training (though probably for a limited part of your lesson, rather than the whole thing). Note also that teaching the content is an essential intermediate step in the process: the importance of feedback gathered while teaching the lesson will be a common refrain throughout this training.
Your Lessons
This training will provide many opportunities for discussion of your lessons. Providing some context now for the lessons that you will be creating will help the Trainers and other participants get involved in those discussions and give you feedback as you follow the process.
Discussion (10 minutes)
Share your answers to the following questions in the shared notes, then discuss them with the Trainers, your collaborators, and the other participants.
- What is the topic of the lesson that you plan to develop based on this training?
- Have you created training material on this topic before?
- What is motivating you to create this lesson?
Iterative Development
The Carpentries community develops open source lessons, which can always be updated and may never be finished. A lesson can undergo many iterations before it reaches a relatively stable state. To reflect this, we encourage lesson developers to indicate the status of their lesson by labelling its progress through a lesson life cycle:
Each life cycle stage indicates the level of maturity of a lesson:
- pre-alpha: a first draft of the lesson is still being constructed.
- alpha: the lesson has been/is being taught by the original authors, but has not been fully tested.
- beta: the lesson is ready to be taught by instructors who have not been significantly involved in its developed to this point.
- stable: the lesson has been extensively tested by the authors and others. It can be considered broadly complete and unlikely to undergo any drastic changes without warning.
Although your lessons will probably remain in pre-alpha throughout this training, some of the content will be equally valuable at later stages and we will also point you towards resources to help with testing the lesson and gathering feedback.
Key Points
- We will learn to develop lessons based on the (slightly adapted) Nicholls’ backward lesson design process.
- There can be many reasons to create a new lesson.
- This training will give you a process to follow to ensure your lesson is effective.
Content from Identifying Your Target Audience
Last updated on 2024-06-26 | Edit this page
Overview
Questions
- Why is it so important to think about the target audience early in the process?
- How can you ensure that your lesson reaches the right audience?
Objectives
After completing this episode, participants should be able to…
- Describe the importance of aligning lesson design with the intended audience.
- Compose a list of prior knowledge required to follow a lesson.
Target Audience
Given the limited time in a short-format training, it is vital to define the scope of the lesson, i.e. what people need to know before and what they will know after the lesson. Thinking carefully about the target audience will help you with this and defining desired learning outcomes (the first step of the lesson design). Prominently displaying a description of the target audience will also help attract people with the right motivation and relevant prior knowledge to attend your workshops.
Expertise
One of the most important things we can identify about our target audience is the level of expertise they will already have in relation to the skills taught by your lesson. In The Carpentries Instructor Training curriculum, we describe three different stages of skill acquisition: novice, competent practitioner, and expert and how it directly correlates to the complexity of mental models these different groups have about a domain/topic.
Briefly, the novice is someone who does not know what they do not know, i.e., they do not yet know what the key ideas in the domain are or how they relate, the competent practitioner has enough understanding of the domain/topic for everyday purposes, and the expert is someone who can easily handle situations that are out of the ordinary and can immediately use their prior knowledge or skills when presented with a new problem in the domain.
When designing a new lesson, it is important to think about the level of expertise that you expect learners to arrive with for two reasons:
- It helps to predict what prior knowledge and mental
model learners will have of the lesson domain when they arrive. This
can enable you to make progress quickly by
- working to help learners recall (activate) that prior knowledge1,
- building on the conceptual understanding they already have2 and,
- perhaps most importantly, giving you some idea of what misconceptions they might arrive with. It is vital that misconceptions are identified and corrected early on, before learners try to incorporate new knowledge into a broken mental model. (More on this in Stay On Target.)
- People at different stages of this process need to be taught differently. For example, novices will learn more from lessons that include worked examples and are more tutorial-like i.e. focused on a specific task, with step-by-step explanations of the process, but without a lot of extra information that is not directly relevant. However, the same approach may actually hinder learning for competent practitioners who may be distracted by a step-by-step explanation of something they already have the prior knowledge of3. For learners at this level of expertise, lessons which include activities offering learners the freedom to explore options and develop their own solutions, are likely to be more effective.
Motivation
Furthermore, your lesson will be more effective if it aligns with the motivations of the target audience. Understanding the wants and needs of your target audience, what they know already and what kinds of problems they want to solve, will help you design a lesson that learners can see the value in. It will give them the impression that taking the lesson will be worthwhile (called positive expectancies in the literature).
We will look more at strategies to establish value and build positive expectancies in the next episode.
Be Specific
It can be tempting to identify a target audience only in vague terms, for example by writing that a lesson is aimed at “PhD students” or “early career researchers”. However, taking the time to focus on real people, or imagined personae, who represent your target audience will help you take time to consider the various aspects that can influence how much someone will learn from your lesson. It will also help you notice when the assumptions you are making about your target audience are unreasonable.
Most of all, it will help you stay connected to the fact that you are not your learners: they will arrive at the lesson with different priorities, interests, and challenges than your own.
Lesson Design Notes Document
At this point, the Trainers will share the template for a Lesson Design Notes document with trainees. Trainees should make a copy of this document and fill in their lesson title. You will populate other parts of the document with the notes and information they produce throughout the training.
Exercise: thinking about target audience (15 minutes total for both parts)
Part 1 (all, 5 minutes): think about a member of the target audience for your lesson, and answer the following questions in the context of your lesson topic:
- What is their background?
- What do they already know how to do?
- What do they want to do with the skills they will learn from your lesson?
- What problem will your lesson help them solve?
Share your answers with your collaborators. How do they compare? If you have identified different audiences, are they compatible? Or would your time be better spent focussing on one particular audience for this lesson?
Take notes on your discussion in your Lesson Design Notes document. It can be particularly helpful to note down any decisions made e.g. potential target audiences that were explicitly discounted, and your reasons for doing so.
Take notes about your choice of target audience in your Lesson Design Notes document. It can be particularly helpful to note down any decisions made e.g. potential target audiences that were explicitly discounted, and your reasons for doing so.
Then write 1-2 diagnostic questions, for use before the lesson is taught, to help you assess whether a respondent falls within the intended audience for your lesson.
There is more to consider about your target audience than we could capture with only the questions listed above. In your own time, you should think more about the other considerations you might need to make when writing a lesson for your audience.
For example, what vocabulary do they use? The terms you are teaching in your lesson might have a different meaning in your learners’ domain of expertise, and it can be helpful to prepare for and try to avoid confusion arising from this clash. Furthermore, might their primary language differ from yours? If so, how might this change the way you write the lesson?
Defining Prerequisite Knowledge
A very common challenge encountered in workshops is heterogeneity of expertise among the audience. When learners arrive at a workshop with a wide range of previous experience with the topic, it is difficult for the instructors to keep everyone engaged. Those who arrive with too little relevant knowledge and experience can struggle to follow the lesson content at the pace you expect, while those who arrive with too much are likely to become bored and despondent as their expectation of learning new skills is not met.
One way to try to guard against this is to publish the description of your target audience when you advertise a workshop teaching your lesson, alongside a list summarising the skills and conceptual knowledge you expect learners to arrive with. Another is to use the information you have about your target audience to ask questions of potential learners when they apply/register to join the workshop (like the diagnostic questionnaire you may have prepared in the exercise above), and use the answers they give to filter out those who fall outside your intended audience.
While valuable, this kind of pre-assessment should be approached with caution: people are often bad at self-assessment i.e. estimating our own ability to perform a task4. We can try to mitigate for this when designing the questions for a pre-workshop survey, leaving little room for inaccurate self-assessment to confound the results. But experience suggests it is very difficult to ensure that every learner in a workshop falls within the intended audience of a lesson.
Exercise: defining prerequisite knowledge (5 minutes)
Write a list of the skills/knowledge your learners will be required to have before they can follow your lesson.
If you are struggling with this exercise because your lesson audience is novices, think about skills like touch typing, using a web browser, or interacting with a command line or graphical interface. These are skills commonly overlooked by experts and competent practitioners.
Key Points
- We recommend an iterative lesson design process that begins with identifying the target audience, before defining learning outcomes, then creating assessments, writing explanatory content, and evaluating the lesson in a workshop.
- Thinking about the target audience early in the design process helps to ensure that your lesson is built around the needs and motivations of real people.
- Use the description of your target audience to help attract people with the appropriate interests and prior knowledge to your lesson.
See chapter 1, How Does Students’ Prior Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
See chapter 2, How Does the Way Students Organize Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
Content from Defining Lesson Objectives/Outcomes
Last updated on 2024-10-17 | Edit this page
Overview
Questions
- How can describing the things you intend to teach aid the process of writing a lesson?
- How can you be specific and realistic about what you will teach in your lesson?
- What are some of the risks associated with unrealistic or undefined expectations of a lesson?
Objectives
After completing this episode, participants should be able to…
- Explain the importance of defining specific, measurable, attainable, relevant and time-bound objectives for a lesson.
- Evaluate a written lesson objective according to these criteria.
At this stage of the training, you should have a clear idea of who the target audience is for your lesson, and what knowledge, skills, and abilities you expect them to arrive with. Now it is time to consider the additional knowledge, skills, and abilities they will have by the time they leave: these are the learning outcomes of your lesson. It can feel strange to jump from one end of the process to the other like this, but clearly defining your goals early in the lesson development process is vital. As we will see in this episode, it helps you to determine the activities, examples, etc. that are appropriate for the lesson content, and provides a scope for what should and should not be included.
Why Focus on Skills?
To ensure your audience stays motivated, and your lesson feels relevant to them, we recommend that lessons focus on teaching skills rather than tools. Lessons should be centred around what you are empowering learners to do, what will be most beneficial to them, rather than a list of functions or commands you are teaching them to use. Placing the emphasis on skills over tools will help you prioritise key concepts and consider how your lesson can have the biggest impact on the way learners do their work.
Learning Objectives
The desired outcomes (the learning objectives) of a lesson should be new skills, i.e. things that the learner can do. For the vast majority of lessons, these will be cognitive skills: things learners can do with their minds. (Lessons intended to teach other kinds of skill, such as woodwork, playing a musical instrument, or making sushi, are probably better suited to a different platform than a static website.) Cognitive skills cannot be equally easily acquired: before we can apply concepts and create something new, we must attain the ability to remember and distinguish between new concepts. Remembering and distinguishing are also abilities that are often faster to gain then applying or creating.
We must try to be realistic about how far along this scale we can move learners during a single workshop/lesson. This is one reason why the target audience is so important: if we can predict what learners will know when they arrive at the lesson, we can better define the outcomes we can expect when they leave.
Defining objectives for a lesson is essential because it allows us to focus the rest of our time on developing content that is necessary for learners to reach these goals. It will help us ensure we do not miss anything important or, conversely, include anything superfluous that could use up valuable time or distract instructors and learners.
What Does an Objective Look Like?
Objectives can be defined for a lesson as a whole - what should learners be able to do at the end of a workshop teaching this lesson? - and for individual sections within it - what should learners be able to do after following this particular part of the lesson?
This is probably a episode-level objective because it is very specific. A corresponding lesson-level objective might be “read data and libaries into the python environment”. It is still fairly specific but a bit broader in scope and contains several episode-level objectives.
The lesson level objective for the current section of this training is: define SMART learning objectives. This objective also covers a later episode as well.
The episode level objectives for the current section of this training are:
Objectives for This Section
- Explain the importance of defining specific, measurable, attainable, relevant and time-bound objectives for a lesson.
- Evaluate a written lesson objective according to these criteria.
These should be read as if they were endings to a sentence beginning
“At the end of this session, learners should be able to …”
Each objective starts with a verb and describes one (and only one) skill the learner will obtain.
For objectives to be as helpful as possible, they need to be written in a way that will allow us to directly observe whether or not a learner has attained the skills we want them to. This means that the skills described by our objectives should be measurable: as a general rule, action verbs such as “explain,” “choose,” or “predict,” are more helpful than passive verbs such as “know,” “understand,” or “appreciate”, which are hard to directly assess and are often open to interpretation.
A popular aid for defining learning objectives is Bloom’s Taxonomy, which divides cognitive skills into several categories. The original taxonomy arranged these categories in a strict hierarchy, which has since been disputed. Regardless of whether these skills conform to such a hierarchy, Bloom’s Taxonomy serves as a very useful bank of action verbs for use in learning objectives. The Committee for Computing Education in Community Colleges has also created a version of Bloom’s Taxonomy customized for computing related training.
We will see how helpful it can be to use action verbs in learning objectives when we begin talking about exercise design in the coming episodes.
SMART Objectives
To assist you in defining and writing learning objectives for your lesson, it can be helpful to turn to a popular framework for defining goals: SMART. Originally proposed to aid managers in the definition of business goals, and updated and adapted since to several other domains including education (see How to Write Well-Defined Learning Objectives for example1), the SMART acronym requires goals to be specific, measurable, attainable, relevant, and time-bound.
In the context of a lesson, SMART objectives should be:
- Specific: they should clearly describe a particular skill or ability the learner should have.
- Measurable: it should be possible to observe and ascertain when the learner has learned the skill/abilities described in the objectives.
- Attainable: the learner should realistically be able to acquire the skills or abilities in the time available in a workshop/by following the text of the lesson.
- Relevant: they should be relevant to the overall topic or domain of the lesson as a whole.
- Time-bound: they should include some timeframe within which the goal will be reached. For learning objectives, this is built into the approach described above.
Note that, for any lesson/curriculum that will be taught in a fixed amount of time, attainable and time-bound are overlapping: learning objectives for your lesson will answer the question “What will learners be able to do by the end of this lesson?” and the time available to teach the lesson, combined with the expected prior knowledge of your target audience, will determine how attainable they are.
Exercise: evaluating learning objectives (15 minutes)
Look at the example learning objectives below. Fill in the table for each objective, checking off the cells if you think an objective meets the criteria or leaving it unchecked if not. You should assume each objective is for a lesson to be taught in a two-day workshop. Note down any observations you make as you move through the list. If you have time, try to imagine the titles of lessons that would have these objectives. This part of the exercise should take 10 minutes.
At the end of this lesson, learners will be able to:
- create formatted page content with Markdown.
- program with Rust.
- fully understand GitHub Actions.
| Objective | Action verb? | Specific | Measurable | Attainable |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 |
In the last five minutes of the exercise, we will discuss as a group how each objective might be improved.
| Objective | Action verb? | Specific | Measurable | Attainable |
|---|---|---|---|---|
| 1 | ✅ | ❓ | ❓ | ✅ |
| 2 | ❓ | ❌ | ❌ | ❓ |
| 3 | ❌ | ❌ | ❌ | ❓ |
Objective 1 is the closest to what we ideally want in a lesson objective, but it illustrates how difficult it can be to make an objective truly specific. For example, a more specific and measurable version of this objective could be:
write links, headings and bold and italicised text with Markdown.
Lesson Scope
One of the major challenges of lesson design is choosing what to include in a lesson: what the main points will be, in what order they will be introduced, how much detail can be provided, and how much time can be spent on each point. Especially when writing lessons for short form training like a Carpentries workshop, difficult decisions often need to be made about what can and cannot be included. Trying to fit too much content into a lesson is counter-productive2, so it is good to avoid the temptation to cram in more content than you have time to cover properly.
Writing learning objectives is a good opportunity to begin thinking about this lesson scope, and can provide assistance when you are faced with a difficult decision about what content to cut out.
For instance, consider the order in which new skills must be acquired. Before learners can begin to acquire “higher-level” cognitive skills to perform creative and analytical tasks, they must first acquire the foundational knowledge and conceptual understanding of the domain. Furthermore, these higher-level skills take longer to acquire so, unless you can expect your target audience to arrive at the lesson with the relevant foundational knowledge and understanding, it is probably unrealistic to aim to have learners completing creative tasks before its end.
As should become clear through activities in the upcoming episodes, lessons can be broadly considered as blocks of content associated with a particular learning objective. This can be helpful when making choices about content to remove, because the task can be considered in the context of taking out whole learning objectives.
Defining Learning Objectives
We have discussed the importance of defining objectives early in the lesson design process, and looked at some examples of objectives written for other lessons. Now it is time to begin defining objectives for your own.
Here are some recommendations to help you get started:
- Aim for 3-4 lesson objectives for every 6 hours your lesson will take to teach. (For example, this curriculum is designed to be taught in ~18 hours and has ten learning objectives.)
- These objectives are for your lesson as a whole: try to define the “end point” knowledge and skills you want learners to acquire. You can think of these as the things you would test in a fictional “final exam” your learners would be able to tackle.
- Later you will write “episode-level” objectives that should define intermediate steps towards the high-level objectives you identify for your lesson here.
Exercise: defining objectives for your lesson (20 minutes)
Write learning objectives for your lesson into the relevant section of your Lesson Design Notes document - what do you want learners to be able to do at the end of the workshop? When writing these lesson-level objectives, try to follow the SMART framework: make them specific, measurable, attainable, relevant, and time-bound.
Take notes about the your discussion in your Lesson Design Notes document. A record of the decisions you made and your reasons for choosing these objectives can be very helpful for you and others to understand the design and scope of the lesson later.
If you find yourself writing very specific episode level objectives, keep them in your notes and try to think of a slightly broader objective that may contain several very specific objectives. We will come back and work more on episode objectives next in the training.
Exercise: reviewing lesson objectives (15 minutes)
Swap objectives written in the previous exercise with a partner (you can also explain or show them what you wrote about your target audience, but this not essential) and review them with the following questions in mind:
- Are the objectives clear?
- Do they use “action” verbs?
- Could you directly observe whether a learner had reached this objective?
Now run the objectives through this Lesson Objective Advisor tool from the University of Manchester’s Faculty of Science and Engineering. Do the results match your assessment?
- Where do the skills described in these objectives sit on the scale?
- (optional) Are these objectives realistic, given the target audience of the lesson?
Advertising your lesson
These learning objectives, as well as the list of prerequisite knowledge you defined earlier, are very useful information to include when advertising a workshop that will teach your lesson. It will help people understand whether or not the event is a good fit for them, and manage their expectations about what they will learn if they attend.
Key Points
- Defining objectives for a lesson can help to focus your content on the most important outcomes, and outline the scope of the project.
- Following the SMART framework can help make your learning objectives as useful as possible.
- Leaving objectives unrealistic or undefined increases the risk of a lesson losing focus or spending time on activities that do not help learners gain the most important skills.
Content from Example Data and Narrative
Last updated on 2024-09-09 | Edit this page
Overview
Questions
- Why should a lesson tell a story?
- What considerations are there when choosing an example dataset for a lesson?
- Where can I find openly-licensed, published data to use in a lesson?
Objectives
After completing this episode, participants should be able to…
- Find candidate datasets to use in a lesson.
- Evaluate the suitability of a dataset to be used in a lesson.
- Choose examples that will prepare learners for formative assessments in the lesson.
- Develop a story
With your high-level lesson objectives set, now is a good time to consider any additional resources you may need to effectively communicate your message to learners before you dive deeper into writing the lesson content.
Creating a Narrative
Writing your lesson as a story helps learners stay motivated and engaged. The story you create can also help learners more easily connect how the skills they are learning now could be useful after the workshop. You can enable learners to make connections between what they learn in your lesson and their own work by creating a narrative that resembles a situation the learners might encounter there.
Here are some example narratives from lessons:
- Software Carpentry Git lesson uses the story of Wolfman and Dracula who have been hired by Universal Missions (a space services spinoff from Euphoric State University) to investigate if it is possible to send their next planetary lander to Mars. They want to be able to work on the plans at the same time, but they have run into problems doing this in the past. They now need a system that will enable them to work collaboratibely and in parallel and track each change.
- Building Websites With Jekyll and GitHub lesson uses the narrative of creating a website for a research project to teach authoring webpages with Markdown and compiling them into a website served on GitHub.
- FAIR Research Software lesson uses the narrative of a poorly designed software project (which analyses NASA’s open data on spacewalks undertaken by astronauts from 1965 to 2013) that over the course of the lesson gets improved in terms of its accessibility, readability, interoperability and reusability to become more FAIR
Sometimes, a single and continuous narrative threads through the entire lesson, while other times, you use a mini-narrative to teach a specific point in an episode. For example, within the overall narrative of creating a research project website in Building Websites With Jekyll and GitHub lesson, there is a mini-narrative of creating a series of short blog posts to teach how to extract a class of structurally similar pages into a common layout template and filter out common structural components of pages.
You may find that images and figures enhance your narrative. Images are powerful communicators, conveying a lot of information with few or no words. Images can also be distracting for learners. If you choose to include images as part of the narrative of your lesson, be sure that they are consistent with the situation you are describing so that they do not increase learners’ cognitive load.
Depending on the tool you are teaching, you might also include a particular dataset as a part of the story you are weaving into your lesson. It is common for lessons to include a dataset that is used in examples and discussed throughout. This can help you maintain a narrative flow and make the lesson feel more authentic. Even if your topic doesn’t require a dataset, deciding on a consistent narrative will help create a flow between episodes and reduce cognitive load for learners.
Use varied examples if you want to teach abstract concepts.
– Neil Brown
From How Your Mind Learns to Program, a lightning talk in the Never Work In Theory series
The focus of this episode is on finding one central example to be used throughout a lesson. However, where you wish for learners to develop understanding of more abstract concepts, you should try to provide a variety of examples with a common theme. This could be achieved by providing multiple opportunities to recognise and apply the concept in taught examples, activities, and exercises.
Finding Images
Copying an image from a website is technologically simple but can be legally and ethically complex. Images are intellectual property and are subject to intellectual property laws including, but not limited to, copyright and trademark laws. These laws differ by country but are consistent in theme: do not take intellectual property that does not belong to you without permission.
When looking for images that illustrate the narrative of your lesson, avoid copying images that do not include a reuse license. Assume that you cannot reuse these images unless you seek written permission from the image creator or owner. Instead, look for images that indicate that they are in the public domain or carry a permissive reuse license such as CC0 or CC-BY. Public domain images can be freely reused and adapted. Images carrying a reuse license can be used and adapted in accordance with their license terms.
If you cannot find reusable images that match your narrative, you can create your own images or seek help from others in the Carpentries community. When incorporating original images into your lesson, be sure to license these images to be compatible with the license on the rest of your lesson materials.
The guidance in this section is not a substitute for legal advice.
Compatible Licenses
Creative Commons offers a chart that identifies which CC licenses are compatible with each other for adaptation (“remix”) purposes: Creative Commons license comparison chart.
GNU offers commentary about a variety of licenses for free software; this resource may be valuable when considering a license for code: GNU: Various Licenses and Comments about Them.
Finding a Dataset
Testimonial
It was very distracting to me in a workshop, when we were given a dataset to work with but no-one explained what that data was and what it meant. It prevented me from following what the instructors were actually trying to teach me, while I tried to figure out the meaning of the numbers, columns names etc on my own.
When searching for a dataset to use in your lesson, there are a number of factors to consider. First, if possible you want the dataset to be an authentic use case for researchers in your target audience. This means balancing the size and complexity of a dataset while avoiding additional cognative load for learners. The dataset may need a certain number of observations to demonstrate some of the skills you are teaching or have a number of variables that is sufficiently similar to what learners may encounter in their work to feel authentic. At the same time, you want your learners to be able to interpret the data fairly easily during the lesson so they can focus on the skills you are teaching. This may mean you will need to subset your dataset by removing some observations (rows) or variables observed (columns).
You may also want to include and review a data dictionary in your lesson, explicitly taking the time to review the information included in the dataset. For inspiration, see the Social Sciences Data Carpentry data dictionary. An additional factor to consider when choosing a dataset to include is the license. You want to find a dataset where the data provider allows for you to freely use it. The best option is a dataset with a CC0 (Public Domain Dedication) license, as other licenses may have more ambiguity around data reuse. Lessons included in The Carpentries Incubator are encouraged to use CC0 licensed data, and may be required to do so to qualify for peer review in The Carpentries Lab. Even with a CC0 license, you will still want to follow best practice in giving attribution to the data provider or collecting agency.
If you’d like to read more about CC0 and CC-BY, Katie Fortney wrote an excellent blogpost on why CC-BY is not always a good fit.
Here are some example datasets from lessons:
- The Ecology Data Carpentry lessons’ dataset comes from the Portal Project Teaching Database. This dataset is an actual ecological research project’s data that was simplified for teaching. The reuse of this dataset throughout the Data Carpentry Ecology lessons helps stitch together the process of data analysis throughout the workshop, from data entry and cleaning to analysis and visualization.
- The Social Sciences Data Carpentry lessons’ dataset is the teaching version of the full Studying African Farmer-Led Irrigation (SAFI) dataset. The SAFI dataset represents interviews of farmers in Mozambique and Tanzania, conducted between November 2016 and June 2017. The interviews surveyed household features (e.g. construction materials used for dwellings, number of household members), agricultural practices (e.g. water usage) and assets (e.g. number and types of livestock). The teaching version of the SAFI dataset has been simplified and intentionally “messed up”” to enable demonstrating common data cleaning issues often found in real-life data.
- Patient inflammation dataset - from the Software Carpentry Python novice and the incubating intermediate lessons - is used to study the effect of a new treatment for arthritis by analysing the inflammation levels in patients who have been given this treatment.
- A river catchment dataset from the Lowland Catchment Research (LOCAR) Datasets is used in the Earth and Environmental Sciences Intermediate Python lesson to analyse hydrological, hydrogeological, geomorphological and ecological interactions within permeable catchment systems.
- Data Carpentry’s Astronomical Data Science with Python lesson uses two astronomical datasets, from the Gaia satellite and the Pan-STARRS photometric survey, to reproduce part of an analysis described in a published article.
When deciding on a dataset, it is also important to consider the ethical use of each dataset. Does the data contain personally identifiable information? Was the data collected without permission from the groups or individuals included? Will the data be upsetting to learners in the workshop? If the answer to any of these questions might be yes, you will need to do more research on it and continue to look for other options. Commonly data is misused from historically excluded and exploited groups. The CARE Principles for Indigenous Data Governance1 (Collective Benefit, Authority to Control, Responsibility, and Ethics) are good starting point for thinking about data sovereignty and considering the ethics of data collected about an individual or groups of people.
Examples of Public Repositories
When looking for data to reuse, consider public repositories in the subject area for your lesson or general data repositories such as:
- Dryad
- The Data Curation Network’s datasets
- The Offical Portal for European Data
- DataONE
- The Official Portal for Argentina Data (in Spanish)
- LANFRICA
Exercise: Choosing a Dataset or Narrative (30 minutes)
Referring to the advice given above, find an appropriate dataset or a narrative for your lesson. Identify one or more potential candidates and note down the advantages and disadvantages of each one. As a reminder, here are some aspects we suggest that you consider:
- For datasets:
- size
- complexity
- “messiness”/noise
- relevance to target audience
- availability
- license
- ethics
- For narratives:
- authenticity
- relevance to target audience
- complexity
- possibility to teach useful things first/early
Takes notes in your Lesson Design Notes document about your discussion and the decisions made. It may be particularly helpful to record:
- Which datasets and narratives did you consider?
- How and why did you choose between them?
- What implications do you think your choice of dataset and/or narrative will have for the design and further implementation of your lesson?
People learn faster if they are motivated2, and learners will be motivated if you teach most useful things first. As you think about what story your lesson will tell, it is important to put the pieces that are most interesting to learners up front. If they are able to quickly learn tools or skills that they see as useful from your lesson, they will be more interested in continuing to learn other concepts that are needed. You may notice this trend in many of the Software Carpentry lessons. In particular, many of the coding language lessons (R and Python) have the learners create a plot very early in the lesson and then go back and teach coding fundamentals such as loops and conditionals. Visualization of data is often a very motivating and much needed skill by learners. Getting to visualization early, keeps learners interested in learning the additional and vital skills where the application might be less clear.
Testimonial
- Dr. Mine Çetinkaya-Rundel
Summary
Remember, even if you do not need a dataset for your lesson, you should decide on a narrative. Centering your lesson around a central example reduces the cognitive load of switching between examples throughout the lesson. Using an authentic, yet simple, dataset will also help reduce cognitive load and help learners to see how they might apply what they learned to their own projects. It is also important to consider licensing and ethical considerations when looking for a lesson dataset.
Key Points
- Using a narrative throughout a lesson helps reduce learner cognitive load
- Choosing a lesson includes considering data license and ethical considerations.
- Openly-licensed datasets can be found in subject area repositories or general data repositories.
The evidence for this is summarised well in chapter 3, What Factors Motivate Students to Learn?, of Ambrose et al. 2010. The Carpentries Instructor Training curriculum also includes a helpful summary of how lesson content can influence Learner motivation.↩︎
Content from Episodes
Last updated on 2024-10-10 | Edit this page
Overview
Questions
- How can the objectives for a lesson be used to break its content into sections?
- How should objectives be written for a smaller part of a whole lesson?
Objectives
After completing this episode, participants should be able to…
- identify appropriate parts of their lesson to break into individual sections
- define learning objectives for a section of a lesson.
So far, you have defined the learning objectives for your lesson as a whole, and came up with a narrative or example datasets that will help you tell your story.
Rather than write the lesson as a single, long document, we recommend that you break it up into chunks like chapters in a book or episodes in a season of a TV series. This can help manage learners’ cognitive load by ensuring that you organise content into coherent, more self-contained chunks, and makes it easier for instructors to schedule regular and frequent breaks while teaching. Thinking about how content can be broken down like this early in the lesson design process helps you to consider the path learners will take to reach your defined objectives, and identify the component skills and knowledge they will need to pick up on the way.
In The Carpentries, we refer to these individual parts of a lesson as episodes, to encourage lesson developers to think about them as self-contained units (with their own narrative arc) that nevertheless contribute to the larger whole (the theme or story that runs through a full season). It also helps to think about the typical length of an episode: these chunks contain 20-60 minutes of content (teaching + exercises).
Each episode will exist as a page in the website we will build for our lessons.
Planning Your Episodes
Before we can start creating the episodes of a lesson, we need to spend some time planning out the number and order of them. The learning objectives you defined for your lesson can help with this: at the very least, it is probably a good idea to have one episode dedicated to each objective. You might also find that you can “decompose” the lesson-level objectives into more finely-grained steps that learners can take towards those end points. For example, the lesson-level objective
- “create formatted page content with Markdown”
may be broken down into,
- “create bold, italic, and linked text with Markdown”
- “explain the different header levels in Markdown”
- “add images with a caption and alternative text description to a Markdown document”
Some questions you might ask yourself to help break down your lesson-level learning objectives include:
- What new knowledge and skills will learners need to acquire to be ready to fulfil the overall objectives for your lesson?
- What order should these concepts and skills be introduced in? Are some dependent on others?
- If some of these concepts and skills are complex, can they be broken down even further?
Exercise: Defining Episodes for a Lesson (25 minutes)
With your team, in the shared notes document for your lesson:
- Based on the lesson-level objectives and your knowledge of the lesson topic, divide the lesson up into logical blocks (episodes), that should each take approximately 20-60 minutes to teach. Think of these logical blocks as topics that you need to cover within your lesson but do not go too deep into defining learning objectives for individual episodes - we will cover that soon.
- Next, assign responsibility for one of these episodes to each collaborator in your team. They will focus on this episode for the rest of this training, and you will teach these episodes in a trial run between parts 1 and 2 of this training.
Some questions you might ask yourself to help break down your lesson-level learning objectives include:
- What new knowledge and skills will learners need to acquire to be ready to fulfil the overall objectives for your lesson?
- What order should these concepts and skills be introduced in? Are some dependent on others?
- If some of these concepts and skills are complex, can they be broken down even further?
If the length of the list of episodes you create is smaller than the number of collaborators you have on your team, try assigning two people to a single episode, with each taking responsibility for a subset of the objectives defined for that episode. If you plan more episodes than you have people in your team, do not assign more than one episode to each collaborator for now, but we strongly recommend that you assign yourselves consecutive episodes at the beginning of your lesson.
Defining Episode-level Learning Objectives
Now that we have broken those lesson-level learning objectives into episodes, we can use the same approach for the individual episodes of a lesson, defining objectives for the episode to make clear what we intend to teach in that section. Defining these objectives before writing the episode content helps us to:
- stay focused in the episode, without spending time on non-essential topics
- determine whether learners are attaining the skills we wish to teach them (we will discuss this more in the next two episodes)
- summarise the skills the learner can expect to gain by following this section of the lesson
As before, here are some recommendations to help you define these episode-level objectives:
- Aim for 2-4 objectives per episode. If you need to write more than that, consider breaking this section down into multiple episodes.
- To ensure your episode content aligns with the overall goals for the lesson, try to identify which lesson-level objective(s) your episode will contribute to.
- In the context of the previous recommendation, consider how the objectives for your episode fit as a component of the higher-level skills/understanding defined in the lesson-level objective(s).
Exercise: define objectives for your episode (30 minutes)
- Using the same approach as you did for your whole-lesson objectives, define a set of SMART objectives for your chosen episode. (15 minutes)
- Compare your list with those created by your collaborators on the
lesson:
- are there any gaps in these objectives, i.e. anything that should be covered in these episodes but is not captured in the objectives?
- are there any overlaps, i.e. anything that looks like it will be covered more than once?
- As a group, discuss how you will address any problems identified in the previous step, and edit your objectives accordingly.
Optional Homework: Reflection Exercise (not included in timing)
Take some time to think back on what has been covered so far, then make some notes on the most important points and actions you want to take away from that. The Trainers and other participants will not look at this - it is only for you.
If you do not know where to start, consider the following list for a starting point:
- draw a concept map, connecting the material
- draw pictures or a comic depicting one of the concepts covered
- write an outline of the topics we covered
- write a paragraph or “journal” entry about your experience of the training today
- write down one thing that struck you the most
Key Points
- Learning objectives for a lesson can help you split up its content into chunks
Content from Designing Exercises
Last updated on 2024-10-17 | Edit this page
Overview
Questions
- How can you measure learners’ progress towards your lesson objectives?
- Why are exercises so important in a lesson?
- What are some different types of exercises, and when should they be used?
- Why should we create assessments before we have written the explanatory content of our lesson?
Objectives
After completing this episode, participants should be able to…
- Describe the importance of regular exercises while a lesson is being taught.
- Choose the format for an exercise based on the outcome it is intended to measure.
As we have seen previously, defining objectives for a lesson (or a teaching episode) can help to focus your content on the most important learning outcomes and outline the scope of your lesson project. The goal of the remaining steps of lesson development is to ensure that what learners learn from following your lesson matches its defined objectives as closely as possible. To do so, you need to develop assessments to monitor progression towards your learning outcomes.
Useful further reading: Course design: Considerations for Trainers – a Professional Guide.
Assessments
In order to measure progress and evaluate if and what learning occurred, we have various types of assessment available to us:
- summative assessments - used to verify whether learners achieved the stated learning objectives after instruction.
- formative assessments - used to detect changes in learner understanding or performance during instruction, to provide feedback and insight (to both instructors and learners) into the learners’ developing mental models of the topic taught and to identify any old or developing misconceptions.
Summative assessments sum up what learning has been achieved after training (e.g. via exams). They give valuable data about learning attainment by individuals and entire cohorts but are not used to guide further progress of a lesson. They may not be as suitable for short courses, but may be necessary for those that give marks/grades or certificates of completion.
Formative assessments are applied throughout a course and with several different purposes: they provide a way to move new information from working memory to long-term memory; they can inform instructors’ decisions about how to modify instruction to better promote learning; they also inform learners about changes they may need to make to improve their learning. Ideally, they should be used often (e.g. after every 15-20 minutes of teaching), providing opportunities to instructors to change pace and refocus learners’ attention. For short courses, formative assessments are usually more valuable and easier to implement in practice than summative assessments - they need not be complex or time-consuming, just informative enough about learning for both instructors and learners.
The most effective way to test learner understanding and progress is to do such assessments in class - they engage all learners and allow instructors to check learners’ confidence with the content and its delivery, can help you deal with any potential misunderstandings as soon as they arise, and maximise the value of workshop for everyone. Such formative assessments also help with metacognition - the awareness a learner has that they are succeeding in learning something new.
Any instructional tool that generates feedback and is used in a formative way to check for learners’ understanding can be described as “formative assessment”. For example,
- checking in - gauging learners’ satisfaction and understanding using agreed signals (e.g. raising different coloured post-it/sticky notes or Zoom reactions to indicate that the pace is too fast/slow, that they completed/have not completed an exercise).
- group or collaborative discussions - raising questions for discussion among the group. For example, think, pair, share, where learners think individually about an answer to a question, pair up with a classmate to discuss their answer, and then share out the consensus they came to with the class.
- problem-solving or diagnostic exercises - setting (coding or non-coding) challenges for learners to tackle, testing comprehension of a subject and diagnosing any forming misconceptions (e.g. via multiple choice questions).
- individual (guided) reflections at the end of a session to help process what was learned, for example asking learners to write down, draw, or diagram (concept map) things they learned, noting how things relate to one another or connect with previous knowledge, things they want to know more about and any questions they still have.
Many other formative assessment tools can be found in Briggs’ list of “21 ways to check for student understanding” or Edutopia’s “56 Examples of Formative Assessment”.
Formative Assessments in this Training (5 mins)
Think back through the parts of this training you have followed so far. Identify two examples of formative assessment that the Trainers have employed so far. As an extra challenge, try to decide whether these assessments were used to assess progress towards a particular learning objective and, if so, what the relevant objective might have been.
Some examples of formative assessments used so far in this training:
- Exercises such as the one asking trainees to describe the target
audience of their lesson
- this exercise aims to assess how well trainees are able to identify the aspects of a target audience that influence the design of a lesson. It also aims to expose any inconsistencies between the visions of the target audience held by different collaborators.
- Your trainers have probably checked in with the group at various
points in the training.
- Although these check-ins are not specific to a particular objective, they help give us an impression of how well trainees are able to follow what we are teaching.
- Tracking the progress of your lesson
repository configuration on GitHub.
- This helps us evaluate trainees’ progress towards the learning objectives we have set in relation to the lesson infrastructure.
Exercise Learners’ Memory
In a simplified model of memory individuals are equipped with two types of memory: working (also called short-term) and long-term. Long-term memory essentially has unlimited storage but is slow to access, whereas working memory is quicker to access but can only hold a limited number of items at a time. For teaching, the goal is to help learners move the new things they’ve learned from working memory into long-term memory. Exercises help with this by providing learners an opportunity to practice what was recently learned. Exercises should occur frequently throughout the lesson to free up working memory and make space for more new information.
Creating exercises builds upon the learning objectives you created earlier in the lesson design process. You can design exercises based on the actions/skills you described in your learning objectives (the learning outcomes you intend for the lesson). This will be easier if your wrote learning objectives with specific action verbs. Specific verbs can help you decide what action you want the learners to perform in the exercise. E.g. actions such as “explain” and “describe” may be better assessed by discussions and multiple choice questions, while “solve,” “construct,” “test” and other higher-level cognitive skills may be better assessed by debugging tasks, code-and-run, or use-in-a-different-context exercises.
Exercise: Exercise Types and When to Use Them (15 minutes)
The Trainers will assign you to pairs or small groups, and give each group an exercise type to work on. Each group should assign a notetaker, to summarise your discussion at the end of the exercise.
The Trainers will assign your group a type of exercise to focus on. Read about your given exercise type in the Exercise Types chapter of Teaching Tech Together by following the relevant link below.
(A Spanish version of Teaching Tech Together is also available.)
Then, considering the exercise type in general, as opposed to the specific example given in the text, discuss the following questions together:
- What kind of skills would an exercise of this type assess? Try to identify some action verbs like those we used to write lesson objectives earlier in the workshop (you may refer to the extended Bloom’s taxonomy for computing to help you).
- Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners?
- Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop?
Share the major points of your discussion in the collaborative notes document.
The Trainers will assign your group a type of exercise to focus on. Read about your given exercise type on the indicated pages of Is This a Trick Question?:
- multiple choice (page 13)
- true-false (pages 20 & 21)
- fill-in-the-blank (page 34)
- authentic assessment (pages 46 & 47)
Then, discuss the following questions together:
- Could exercises of this type be used in your lesson?
- If so, can you identify any of your written objectives that could be assessed with an exercise of this type?
- Have your Trainers set you any exercises of this type in this training so far?
- Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners?
- Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop?
Exercise: Exercise Types and When to Use Them (15 minutes) (continued)
Trainers will lead a discussion about your findings for your chosen exercise type with the rest of the participants.
Both of the resources linked from the exercise above, the Exercise Types chapter of Teaching Tech Together and Is This a Trick Question? are worth reading in full. They collect a lot of insightful discussion and illustrative examples together, which can prove very useful when designing exercises for your lesson.
As you discussed with your group in the last exercise, different types of learning objectives work better for novices, while others are a better fit for competent practitioners or experts.
This can be understood in terms of the types of exercises that suit the objective: exercise types that help manage cognitive load for the learner, such as fill-in-the-blanks, faded examples or Parsons problems (which all provide a lot of the guiding process/scaffolding code and allow the learner to focus on a specific concept or skill) are a good fit for a novice, to whom all elements of the topic are new. However, these kinds of exercise do not provide an opportunity for learners to develop higher-level skills, such as the ability to create whole new functions or scripts, or to extrapolate from the examples they have seen to solve a different kind of problem. Indeed, example and exercise types that are helpful to novices may even be counter-productive for learners with a greater level of expertise1.
Thus you want to choose your objectives to fit your intended audience and your exercise formats to fit your objectives.
An Expert’s View on Objectives and Audience Expertise
“Different types of lesson objectives (LOs) are better fit for novices, while others are better fit for competent practitioners, etc. and if exercises (formative assessments) are well aligned to LOs, [they] will automatically serve the corresponding audience. Thinking in terms of LOs (What should a learner do in order to achieve this specific LO? Is this LO exactly what learners are expected to achieve by the end of this piece of instruction? etc.) is easier than thinking in terms of LOs + audience + content. LOs should be tailored to the audience, and, if this is well done, you may stop worrying about the audience. Create LOs for the specific audience and create assessments for specific LOs.”
- Dr. Allegra Via, Carpentries Instructor Trainer
Detecting and correcting misconceptions and fixing learners’ incorrect/broken mental models is as important as presenting your learners with new knowledge and correct information. When mental models are broken, learning can occur slower than you might expect2. The longer a prior incorrect model is in use, and the more extensively it has to be “unlearned”, the more it can actively interfere with the incorporation of the new correct knowledge (since it will contradict the misconceptions already present in the mental model).
When designed well, multiple choice questions can diagnose misconceptions you predict that learners might have, and help you correct them quickly.
We recommend that you read the sections of the Instructor Training curriculum and Teaching Tech Together linked above. You may also find it interesting and helpful to review our supplementary page digging into misconceptions and multiple choice questions in more detail.
Exercise: Assessing Progress Towards an Objective (30 minutes)
Using one of the exercise formats you have learned about so far, design an exercise that will require learners to perform one of the actions described in the objectives you wrote earlier, and that assesses their ability to do so.
These should be assessments of the lower-level objectives defined for individual episodes in the lesson, as opposed to the lesson-level objectives you wrote first.
Trainees working as a team can choose whether to work together on discussing and designing a single exercise to assess a single objective, or to divide their efforts and each focus on an exercise for their own episode. If you choose to take the latter approach and finish with time to spare, spend the remainder reviewing and providing feedback on one another’s assessments.
The same approach to designing exercises within a lesson can also be used to create a short “pre-assessment” questionnaire for potential learners to complete when they register for a workshop teaching the lesson (or for self-evaluation before following the lesson on their own). You can use the list of prerequisite knowledge that you defined earlier to help with this.
If you collect the results of this questionnaire, use it to follow up with people who have registered for the workshop but do not fit the intended target audience, to manage their expectations about how useful the workshop will be for them.
You should aim to create all your assessments before you have written the explanatory content of your lesson (recall Nicholls’ backward design). These assessments will guide your lesson design process by knowing exactly which knowledge you’d expect from your learners at any point in the lesson.
Well-designed exercises are one of the most valuable resources for an instructor and any time spent on this during lesson design is well invested.
Key Points
- Assessments are a way to determine whether the objectives you defined for the lesson have been reached.
- Exercises help learners commit what they’ve learned to long-term memory.
- Some types of exercises are better for particular audiences and to address certain objectives.
- Formative assessment happens during teaching and provides feedback both to an instructor and a learner - about progress and whether learning of new concepts occurred but also about any misunderstandings and misconceptions which can hinder further learning.
See chapter 1, How Does Students’ Prior Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
See chapter 1, How Does Students’ Prior Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
Content from How to Write a Lesson
Last updated on 2024-10-24 | Edit this page
Overview
Questions
- Why do we write explanatory content last?
- How can I avoid demotivating learners?
- How can I prioritise what to keep and what to cut when a lesson is too long?
Objectives
After completing this episode, participants should be able to…
- Estimate the time required to teach a lesson.
- Summarise the content of a lesson as a set of questions and key points.
- Connect the examples and exercises in a lesson with its learning objectives.
Writing Explanatory Text
Explanatory text (including examples) help connect your exercises together into a cohesive lesson. If your exercises are train stations and scenic views, the explanatory text is the train tracks that connect them. Often times the examples and exercises may seem like a good draft for you to teach. However, the explanatory text makes it possible for other instructors and self-directed learners to follow along by providing all the relevant information directly in the lesson. It can also help you remember your goals and stay on track teaching.
How much text to include is often a question of personal style while balancing the needs of potential users, both other instructors and learners. You want to find the balance between providing enough information for learners to meet the learning objectives without increasing cognitive load with extraneous information. In this section we will discuss factors to consider when you are writing explanatory text.
There are trade-offs to consider when preparing a site for use as an instructional guide vs use as a self-directed learning resource. Typically, self-directed learning resources are more verbose where instructional guides are aimed at an audience that can fill in the gaps on their own. However, very sparse text is less likely to be re-usable by other instructors as instructors may have different skill levels with the lesson content or differing mental models. In general, it is not a good idea to assume others, learners or instructors, will know what you were thinking when you wrote the content so, if in doubt, be explicit.
Exercise: Examples Before Exercises (20 minutes)
Looking back at one of the exercises you designed before: what examples could you include in your narrative to teach learners the skills they will need to apply to complete the formative assessments you have designed?
Outline one of these examples in your lesson design notes.
In the Software Carpentry Plotting and Programming with Python Lesson there is an exercise to load and inspect CSV file for Americas. In the lesson, before the exercise, the Instructor demonstrates an example of how to load the data table for another continent (Oceania) and explores the values with a few different functions. This shows learners how to call the function to load the CSV into a data frame, and demonstrates what success looks like for their exercise.
In the Python Interactive Data Visualization Lesson in the Incubator, there is an exercise to find the correct widget (a slider) for an action and modify the script to use it. In the lesson, before the exercise, the instructor introduces a cheatsheet and documentation for interactive widgets and then creates a dropdown widget for the application. The slider widget required in the exercise has not been demonstrated but the preceding example shows all of the necessary steps to add a widget, and provides the supporting information that learners can consult to discover how to implement the specific tool.
Less is More
Trying to fit too much content into a lesson is counter-productive. It is better to cover less and provide a smaller but stable foundation for learners to build upon.
You should also consider a realistic workshop length for teaching your lesson. It is difficult to keep learners attentive for many hours over many days. Most Carpentries workshops are two work days of lesson material as it is difficult for learners to attend 4 full days of a workshop. Although, there are some in the community who will run workshops that are three days to cover additional material.
As you consider the length of your lesson discuss with your collaborators and ask yourself:
- What is essential to include?
- What can be left out if needed?
- Are there checkpoints where the lesson could end if needed?
- Can important concepts be moved up earlier to ensure they are covered?
In the end, the only way to know for sure is to teach the lesson, measuring how long it takes to teach. Borrowing from television, The Carpentries community calls these early workshops, pilot workshops. As you run pilot workshops, you can note the length of time spent in each episode. You might find the template for notes on pilot workshops helpful as it includes a table for episode and exercise timings. Instructors commonly report running short on time in workshops. Thus, it is better to assume that you are under-estimating rather than over-estimating the length. It is better to have additional time for discussion, review, and wrap-up than to rush through material to fit it into the remaining time.
If, after piloting your new lesson, you find that you did not have time to cover all the content, approach cutting down the lesson by identifying which learning objectives to remove. Then take out the objective(s) and the corresponding assessment(s) and explanatory content. You may consider making a concept map to help identify which objectives depend on one another.
Alternatively, if you decide to keep certain objectives in the lesson, you can add suggestions on which objectives can be skipped to the Instructor Notes for the lesson. Instructor Notes are meant to convey teaching tips and advice to other instructors teaching your lesson and will be discussed in more detail later on in section “Preparing to Teach”.
As you add notes and think about what to cut, remember, reducing the number of lesson objectives can help with managing learner (and instructor) cognitive load.
Testimonial
“the five ways to handle an extraneous overload situation are to:
- eliminate extraneous material (coherence principle),
- insert signals emphasizing the essential material (signaling principle),
- eliminate redundant printed text (redundancy principle),
- place printed text next to corresponding parts of graphics (spatial contiguity principle), and
- eliminate the need to hold essential material in working memory for long periods of time (temporal contiguity principle).”
Lesson Time Management (10 minutes)
(5 minutes) In the shared notes document, note down your answers to these questions:
- From a design perspective, at what point is a lesson too long?
- What factors influence and constrain the length of a lesson?
- How might you prioritise what to keep if you have to cut lesson content down?
(5 minutes) In the remaining time, your Trainers will lead a discussion based on the responses.
Other Important Considerations for Lesson Content
Language
When writing your lesson text you also want to avoid unintentionally demotivating your learners. Make time to review your text for:
- dismissive language - e.g. ‘simply’, ‘just’
- use of stereotypes (check learner profiles for stereotypes too)
- expert awareness gaps, i.e. places where you may be assuming the learners know more than they actually do
- fluid representations, i.e. using different terms with the same meaning interchangeably
- unexplained or unnecessary jargon/terminology (as your learners may come from different backgrounds, may be novices, not native English speakers, and a term in one domain/topic may mean something else entirely in another)
- unexplained assumptions
- sudden jumps in difficulty/complexity
Accessibility
You should also review your text thinking about accessibility. This includes:
- Avoiding regional/cultural references and idioms that will not translate across borders/cultures
- Avoiding contractions i.e. don’t, can’t, won’t etc.
- Checking that all figures/images have well written alternative text, including writing alternative text for data visualizations. An alternative text contains an invisible description of an image which is read aloud to blind users by screen readers providing alternative text-based format for images. See the Workbench documentation for directions on how to add alternative text and captions to images in a lesson
- Checking the header hierarchy - no h1 headers in the lesson body, no skipped levels
- Using descriptive link text - no “click here” or “this page”, etc.
- Checking the text and foreground contrast for images
Optional Exercise: Alternative Text for Images (5 minutes)
Which of the following is a good alt-text option for the image below?
- Graph of data
- Graph with increasing lines
- Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory from 1958 to present
- Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory, Hawaii, United States, from 1959 to present including values from each year. Red line shows variation in each year and black line is average for each year. 1959 = 315.90 ppm, 1960 = 316.91, 1961 = 317.64 …
Data/Image provided by NOAA Global Monitoring Laboratory, Boulder, Colorado, USA
The third option, “Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory from 1958 to present”, is the best option. It describes the type of plot, what is measured, and the trend.
The first two options are too vague. They mention a graph but don’t give enough info to know what the graph is actually showing.
The last option is overly descriptive and starts to list the data itself. It is better to include a shorter, but informative, description and a link to the data instead.
Glossary of Terms
When writing a lesson, you should also consider adding key terms or jargon to the lesson glossary for the lesson. In your shared notes document, you can start such a glossary as you develop your lesson, to keep a record of the terminology your audience needs to be familiar with.
Many of these terms may also be useful for other lessons and can be added to Glosario, a multilingual glossary for computing and data science terms.
Exercise: Explain Your Terminology (5 minutes)
In your shared notes document, add a list of terms or jargon from your lesson, along with their definitions.
You do not have to have a complete list now, but it is good to start working on it - you can always come back and update it later.
Questions and Keypoints
In addition to objectives, a completed episode also requires questions the episode answers and the keypoints a lesson covers.
The questions helps learners understand what to expect from a lesson as they might not yet understand the learning objectives.
The keypoints wrap up the lesson by providing answers to each of the questions. Keypoints also help self-directed learners review what they learned, remind instructors to recap what was covered in an episode, and help you ensure you answered all the questions you intended for an episode.
Exercise: Completing episode metadata (10 minutes)
In your shared notes document, add key points and questions for your episode.
Polishing a lesson
Lessons do not need to be perfected prior to teaching them the first time. In fact, keeping things relatively basic leaves you some flexibility while you run through the first few iterations of teaching the lesson, collecting feedback, and using that to guide improvements.
However, once you find that the lesson is beginning to reach something like a stable state, you may find the checklist for lesson reviewers in The Carpentries Lab useful as a guide for polishing the design and content. The checklist describes criteria for a lesson to meet a high standard in terms of its accessibility, design, content, and supporting information.
Key Points
- The objectives and assessments provide a good outline for an episode and then the text fills in the gaps to support anyone learning or teaching from the lesson.
- It is important to review your lesson for demotivating language, cognitive load, and accessibility.
- To reduce cognitive load and ensure there is enough time for for the materials, consider which lesson objectives are not needed and remove related content and assessments.
Content from The Carpentries Workbench
Last updated on 2024-11-07 | Edit this page
Overview
Questions
- How is a lesson site set up and published with GitHub?
- What are the essential configuration steps for a new lesson?
Objectives
After completing this episode, participants should be able to…
- Identify the key tools used in The Carpentries lesson infrastructure.
- Complete the fundamental setup steps for a new lesson repository.
- Edit Markdown files using the GitHub web interface.
At this stage in the training, you will have gone through the first three stages of the lesson design. You should have a clear idea of who the people are that you want to teach this lesson to, and what exactly the skills are that you want them to learn while they are following it. You also have an outline of your lesson and its episodes. It is now time to begin creating a website that will present that lesson to the world!
GitHub Pages
The source of all The Carpentries lessons is made publicly-available in repositories on GitHub. By making our repositories public like this, we encourage others to help us maintain and improve our lessons, and make it as easy as possible for them to re-use and modify our lessons for their own purposes.
GitHub provides a hosting service to open source projects such as these, allowing users to present their projects to the wider world. The repository includes a complete history of the changes made between versions of the individual files in the project, and provides many features that facilitate collaboration on projects. We will learn more about some of those collaborative features later in this training. For now, we will focus on one other important feature that GitHub provides: website hosting.
Via a system called GitHub Pages, users can build and host websites from the files present in any repository on GitHub. For many years, this has been how The Carpentries presents its lesson websites to the world.
Using The Carpentries Workbench
Carpentries lesson websites are built with The Carpentries Workbench, a toolkit that converts Markdown and R Markdown files into the HTML that can be served by GitHub Pages. We will use it now to initialise a new lesson.
In the past, our lesson sites were generated by software called Jekyll, a tool built into GitHub Pages that allows the webpage content, written in the text files of the repository, to be combined with descriptions of settings, structure, and styling, to create a website. The template used by Jekyll to structure and style lessons was initially developed in 2013/2014 by Abby Cabunoc Mayes, Greg Wilson, Jon Pipitone, and Michael Hansen for Software Carpentry. It was expanded and maintained by many members of the community over almost a decade, to also support Data Carpentry, Library Carpentry, and many other lessons.
In 2022, we adopted a new infrastructure for our lesson sites: The Carpentries Workbench. Lesson sites built on the Workbench are still hosted with GitHub Pages, but no longer use Jekyll. Instead, the lessons are built using a programming language, R, and pandoc, a software designed for converting content between file formats. The Workbench combines three R packages:
- sandpaper: converts a collection of Markdown or R Markdown files into the structure of a lesson website.
- varnish: provides the files and folders that add styling and additional functionality to a lesson website.
- pegboard: a programmatic interface to the lesson, enabling various automated validation tasks.
For lesson developers, the Workbench makes The Carpentries lesson repositories much simpler to navigate and work with.
Developing lessons using the Carpentries Workbench in GitHub might seem intimidating, especially for those who are less experienced with collaborative GitHub workflows or unfamiliar with the Workbench. However, most of the complex infrastructure is hidden from lesson developers, and some setup tasks only need to be completed once or infrequently. The table below outlines common tasks that lesson developers perform in GitHub (which you will learn about shortly) along with how often they are needed.
| Task | Frequency |
|---|---|
| Create repository | Once |
| Add collaborators | Rarely |
| Edit global lesson configuration file | Rarely |
| Edit lesson front page | Rarely |
| Create a new episode | Often in the early stages of lesson design, rarely after that |
| Edit episode content | Often |
Creating a Lesson Repository
To get started, we first need to create a new repository for our lesson. We will use a template to do this, so that the new repository contains the basic files and folders that the Workbench needs in order to build a lesson site. There are currently two templates to choose between:
- A Markdown template
- An R Markdown template, best suited to lessons you expect to include R source code that will generate output.
One member of each participating lesson team should choose one of these templates, following the link above and completing the configuration as follows:
- click on the green “Use this template” button near the top right of the window
- add a name for the repository
- The name should be descriptive but fairly brief, with hyphens
(
-) to separate words - the name can always be changed later, via the repository settings
- The name should be descriptive but fairly brief, with hyphens
(
- make sure the “Include all branches” box is checked to copy all
branches from the template repository
- this is important as lessons are developed in branch
mainbut rendered into websites from branchgh-pages
- this is important as lessons are developed in branch
- in the “Description” field, write the title of your lesson
- choose “Public” visibility
After pressing the Create repository button, you should be presented with a brand new lesson repository, like in the picture below.
renv/ directory.Adding collaborators
To be able to add content to the lesson, your collaborators on this project will need access to the repository. To add collaborators to the repository, navigate to Settings, then choose Collaborators from the left sidebar. Now repeat the following steps for every collaborator working with us on the project.
- Click Add people and enter the GitHub username of one of our collaborators.
- Click Add USERNAME to this repository.
Your collaborator should receive an email inviting them to join the repository. After they have accepted this invitation, they should be able to edit the repository, adding new files and modifying existing ones. Only the person who created the repository will be able to adjust the repository settings.
Repository Files
The repository contains a number of files and folders. Most of these are source files for the content of our new lesson, but a few are accompanying files primarily intended for the repository itself rather than the lesson website. These are:
CITATION.cffCODE_OF_CONDUCT.mdCONTRIBUTING.mdLICENSE.mdREADME.md.gitignore- and the
.github/folder.
We will address all of the files later in the training. For now, we will move on to complete the basic setup of the lesson.
Configuring a Lesson Repository
Since your lesson was created from a lesson template which contains
gh-pages branch, GitHub Pages should already be generating
a rendered version of your lesson from this branch. If it isn’t, the
instructor should work with groups to activate GitHub Pages.
Although we configure GitHub Pages to serve the lesson website from
the gh-pages branch, the default working branch for
a lesson will be main. For the rest of this
training, you should add and edit files on main, and in
future, when you open Pull Requests to update the lesson content, these
should also be made to main. The gh-pages
branch should never be modified by anyone other than the automated
actions-user account.
To see the rendering of the gh-pages branch by the
automated actions-user, you can click the “Actions” tab from the main
repo page. This tab shows the actions triggered by each commit and if
they are in-progress or completed (sucessfully or with an error). You
can also see the details of actions in more depth by clicking on an
individual commit in this page.
On the repository home page (e.g. by clicking the name of the project near the top left of the window), click the gear wheel icon near the top right, to edit the About box. Check the box that says “Use your GitHub Pages website” to add the address of your lesson site to the About box.
After following these steps, when you navigate to the pages URL, you should see a lesson website with The Carpentries logo and “Lesson Title” in the top-left corner. You may need to wait a few minutes for the website to be generated.
config.yaml
The lesson title can be adjusted by modifying the
config.yaml file in the repository. The
config.yaml file contains several global parameters for a
lesson - to determine some of the page styling, contact details for the
lesson, etc. config.yaml is written in YAML, a structured file format of
key-value pairs in the form key: value. For example, a YAML
file of personal data might include lines such as
name: 'Mei', height_m: 1.84, and
birthdate: 1899-01-12. As well as the title of the lesson,
you can and should adjust some of the other values in
config.yaml, but you should not need to add new values or
learn a lot about YAML to be able to configure your lesson. You should
also only have to add or modify single-line string values and list
entries, and not deal with multi-line strings or other data types
(e.g. numbers, booleans, dictionaries) that YAML supports. The template
config.yaml aims to guide users through examples and
annotations on how to format the values they provide e.g. if you are
modifying a value wrapped in quotation marks in the template file, it is
safest to replace it with another value within quotation marks.
Practice with config.yaml (5
minutes)
Complete the configuration of your lesson by adjusting the following
fields in config.yaml. Remember that FIXME values with
quotes should be replaced with another value still in quotes and FIXMEs
without quotes should be replaced by values without quotes.
-
contact: add an email address people can contact with questions about the lesson/project. -
created: the date the lesson was created (today’s date) in YYYY-MM-DD format. -
keywords: a (short) comma-separated list of keywords for the lesson, which can help people find your lesson when searching for resources online. At a minimum, include ‘lesson’, ‘pre-alpha’, and your lesson’s (human) language. For example, ‘lesson, pre-alpha, español’. -
source: change this to the URL for your lesson repository.
We will revisit the life_cycle and
carpentry fields in config.yaml later in this
training.
.Rproj file
The repository also contains a file called FIXME.Rproj,
a project file that RStudio (a commonly-used editor for working on
lessons) can use to track some options and preferences for working on
your lesson on your local machine. This file should be renamed. You can
call this file whatever you like, but the convention is for its name to
match that of the lesson repository.
README.md
The README.md file is the “front page” of your lesson repository on GitHub, and is written in Markdown. You should use it to provide basic information about the project, for anyone who finds their way to the source files for your lesson.
Improving the README.md (5
minutes)
Take a few minutes to update it with some basic information about the project:
- the lesson title
- a short description of the lesson
- a list of the names of the authors, optionally linked to their GitHub profile
We will revisit README.md later in the training with
more details on what to include in this file.
Key Points
- Lesson sites are built with GitHub Pages from source repositories located in GitHub.
- A new lesson repository can be created from a template maintained by
The Carpentries, and configured by adjusting the
config.yamlfile.
Content from Adding Lesson Content
Last updated on 2024-10-25 | Edit this page
Overview
Questions
- How do you create and modify the pages in a lesson?
- How should different structural elements be presented in a lesson website?
Objectives
After completing this episode, participants should be able to…
- Add lesson episodes as individual pages of a lesson website.
- Use fenced divs to create different structural elements within a page to format objectives, questions, keypoints, exercises and their solutions in a lesson website.
Now that we have completed the basic configuration of a lesson site, it is time to move on and look at where the actual lesson content will be written.
Lesson Home Page
The “home page” of a lesson is created from the index.md
file: this file should contain a brief introduction to the lesson,
designed to give visitors to the page a first impression of the lesson
and whether it is appropriate for them. The file begins with a short
header, written in the same key-value pair syntax we encountered in
config.yaml.
This header configures how the site will be built by
sandpaper, one of the components of The Carpentries
Workbench, and should be left untouched. The page content begins after
the blank line that follows this header.
You might have noticed by now that GitHub provides a preview of
content in Markdown files, and displays an interpreted (“rendered”)
version of Markdown file content by default in its web interface. You
might also notice that the content of index.md is displayed
differently when viewed on GitHub from how it appears on the homepage of
your lesson website: the header mentioned above appears in a tabular
layout on GitHub, but is not visible on the lesson homepage.
This is because the lesson infrastructure accepts an expanded set of elements in the source Markdown files: syntax for components of a lesson website that are not part of the standard set of Markdown features, e.g. page metadata such as the YAML header above, or blocks to make exercises and solutions visually distinctive. In these cases, you may see GitHub render the content differently from how it appears on your website, or not at all.
We will encounter more examples of this throughout the lesson, but the important thing to remember for now is that how GitHub displays source (R) Markdown files is not a reliable indicator of how content will be displayed in your website.
To get started on this home page, replace the template text with the
same short description you added to your README.md.
Below those, you can add the prerequisite skills you determined
earlier for your lesson, as a bullet point list to
index.md:
Exercise: practice editing Markdown in GitHub (optional)
Add the objectives you defined for your lesson as a bullet list in
the index.md file of your lesson repository.
Episodes
The main body of the lesson is written in episodes: the
individual chunks or sections that the lesson is separated into. The
episode pages of the lesson site will be constructed from Markdown or R
Markdown files in the episodes folder of the lesson
repository.
The episodes folder of the new repository contains a
single file, introduction.md. The content of this file
includes examples of how to write Markdown files for The Carpentries
Workbench.
There are two important things to note:
- Like
index.md, the episode file begins with a short header. The fields in this header describe the episode title and the estimated time (in minutes) required to teach it and complete the exercises. - The example content includes several lines that start with strings
of colons (
:::::::). These mark the beginnings and ends of structural elements within the page, called fenced divs, which get rendered in a visually distinct way on the lesson website. Each fenced div starts and ends with a string of at least 3 colons (“:::”) and the word at the end of the starting colons (the fenced div’s tag) indicates what kind of block it is. There are many types of fenced divs in the lesson infrastructure and we will explore some of them in this episode.
Creating a new episode
Let’s create a new episode file, for one of the episodes you have
just identified. First, open the “raw” view of the
introduction.md example episode, and copy its contents.
MARKDOWN
---
title: "Intro to Markdown"
teaching: 10
exercises: 2
---
:::::::::::::::::::::::::::::::::::::: questions
- How do you write a lesson using Markdown and `{sandpaper}`?
::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::: objectives
- Explain how to use markdown with The Carpentries Workbench
- Demonstrate how to include pieces of code, figures, and nested challenge blocks
::::::::::::::::::::::::::::::::::::::::::::::::
## Introduction
This is a lesson created via The Carpentries Workbench. It is written in
[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
[R Markdown][r-markdown] for dynamic files that can render code into output.
Please refer to the [Introduction to The Carpentries
Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
...
::::::::::::::::::::::::::::::::::::: keypoints
- Use `.md` files for episodes when you want static content
- Use `.Rmd` files for episodes when you need to generate output
- Run `sandpaper::check_lesson()` to identify any issues with your lesson
- Run `sandpaper::build_lesson()` to preview your lesson locally
::::::::::::::::::::::::::::::::::::::::::::::::Now create a new file in the episodes folder. Based on
the episodes you planned out earlier, choose a name for your episode
that concisely describes the intended content,
e.g. data-visualisation.md (or
data-visualisation.Rmd for a lesson based on R Markdown).
It is vital to include the file extension when naming this file: only
files with the .md or .Rmd extensions will be
built into webpages by the lesson infrastructure.
For page content, paste the content the introduction.md
file you copied earlier and:
- replace the title in the header with the title of your episode
- set the
teachingandexercisesfields to zero for now - replace the contents of the
questionsdiv with the questions for your episode you defined earlier - replace the contents of the
objectivesdiv with the episode-level objectives you defined earlier - replace the contents of the
keypointsdiv with the key points for your episode you defined earlier - Ignore or delete the rest of the content of the episode (i.e. only focus on the three structural elements - questions, objectives and keypoints).
MARKDOWN
---
title: "Episode Title"
teaching: 0
exercises: 0
---
:::::::::::::::::::::::::::::::::::::: questions
- question 1
- question 2
- ...
::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::: objectives
- objective 1
- objective 2
- ...
::::::::::::::::::::::::::::::::::::::::::::::::
Some episode content ...
::::::::::::::::::::::::::::::::::::: keypoints
- keypoint 1
- keypoint 2
- ...
::::::::::::::::::::::::::::::::::::::::::::::::The
Workbench Component Guide provides a full list of all the different
classes of fenced div that can be included using the lesson
infrastructure. The guide specifies which divs are required for an
episode (objectives, questions, and
keypoints), and which are optional.
Adding a new episode to the lesson navigation
This new episode will not yet appear in the navigation of your lesson
site. To enable this, we need to specify where the episode should appear
in the order of the lesson. That episode order is defined in the
episodes field of the config.yaml file:
Add the name of the new episode file you created to this list in
config.yaml and commit this change, for example:
After the lesson site has been rebuilt on GitHub, you should see the episode title appear under Chapters in the left sidebar navigation of your lesson site after refreshing the webpage. Clicking on that title will take you to the episode page built from the file you created. At the top of the page body, you will find the episode title and an Overview box containing a list of the questions and objectives defined for the episode. Later, we will add more content to your chosen episodes.
TODO add a labelled screenshot of a new episode page
Exercise: practice creating episodes (10 minutes)
Repeat the steps you just saw, to create another new episode file and add it to the lesson site navigation. If you know what another episode in your lesson will be about, create the page for that. Otherwise, feel free to use any values you like for the file name and episode title.
Using this approach, we can build up our lesson one episode at a time.
Adding Exercises
To create an exercise in The Carpentries Workbench, we use another two types of ‘fenced divs’:
-
discussions, where the main task is for participants to discuss a topic or prompt, and -
challenges, where the main task is a problem to be solved.
For example, to start a challenge fenced div, use at least 3 colons,
followed by the challenge tag. Then the content of the
challenge is included on the following lines. Finally, you need to close
the fenced div using another line with least 3 colons.
MARKDOWN
:::::::::::::::::::::::::::::::::::::: challenge
### Challenge Title
Challenge text, code, and other information goes here
::::::::::::::::::::::::::::::::::::::::::::::::::If you also want to include an expandable solution box for the
challenge you can nest a solution fenced div within the
challenge div. The format is the same as for a challenge
except the fenced div tag is solution instead. Note the
solutions can all be expanded for more accessible reading using the
“Expand All Solutions” option at the top of each episode.
:::::::::::::::::::::::::::::::::::::: challenge
### Challenge Title
Challenge text, code, and other information goes here
:::::::::::::: solution
### Solution Title
Solution text, code, and other information
:::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::
For readability, you may want to have the length of the closing lines match the opening lines. See in the above how the challenge and the nested solution’s closing lines are similar lengths to the their corresponding opening lines. For more information about creating exercises see the Workbench Documentation for Exercises.
Exercise: Formatting Exercises in a Lesson Site (15 minutes)
Using the approach demonstrated above, format the exercise you designed previously as an exercise in your lesson site.
Optional and Important Material
The Carpentries Workbench includes fenced divs for highlighting key material that should not be skipped during instruction, as well as optional content that can be covered if time permits.
Often lessons have more content than can be reasonably taught in the amount of time allotted. This is especially true for collaboratively developed lessons as each contributor/instructor may have additional items they’d like to see included, leading to “scope creep”.
To address this issue, in many Carpentries lessons, spoiler boxes are used for asides and short tangents, e.g. points that might be relevant to some audiences but are not essential to the flow of the lesson. These spoilers should still be kept to a minimum as they can be disruptive to instructors and readers.
Note, this use of the spoiler box is to use the expandable box functionality as nothing will be “spoiled” by expanding this box. Also, note that the spoiler box titles should be very clear so instructors need not expand the spoiler to know if they want to teach that extra.
Calling Attention to Important Points
For important points in the lesson, you can add them to a callout box to emphasize their importance.
You may notice that many of the older lessons use callouts for both additional material and to highlight important points interchangeably. Spoilers were developed in 2023 to help with separating these two different use cases.
Glossary of Terms
You have already started on compiling a glossary of terms for your
lesson. The Workbench offers a standardized location for lesson
terminology: learners/reference.md.
Exercise: More Practice (15 minutes)
Use this time to add more content to your lesson site. Generally speaking, you should try to transfer the drafted content from your lesson design notes into the lesson website. Here are some suggestions for things you might try:
- Add your list of prerequisite knowledge/skills as a fenced div with
the
prereqclass to yourindex.mdfile. - Add some tabbed content in your episode.
- Start a glossary of terms in the
learners/reference.mdfile, referring to the Workbench documentation on how to add a list of term definitions. - Look through the Workbench component guide and try implementing some of the other flavours of fenced div.
- Add a new Markdown file to the
learners/orinstructors/folder and see if you can find the built page in your lesson site.
Troubleshooting the Lesson Build
Sometimes, formatting errors and typos in the files of your repository can cause the process that builds your lesson website to fail.
You will likely first notice the failure to build the lesson if the lesson website is not showing the changes you made. You may also receive an email from GitHub about the build failure if that is your preferred way of receiving notifications (but it may take you some time to realise this). The good thing is that GitHub keeps your previous lesson version online until the error is fixed and a new build is completed successfully. If you are not yet familiar with the GitHub interface, it can be difficult to pinpoint the problem.
Here are some points of advice that you can follow to help find and fix the problem when you notice the build process fail.
- When you first notice that there is a problem, stop editing your lesson: any subsequent changes that you make will not be included in your lesson site until the problem is fixed, and might introduce additional issues, making it more difficult to find the original cause.
- Look at the history of commits (the link with a reversing stopwatch icon and “NN commits” at the top of the listing of files and folders in the repository homepage). Is there a place where the build has a red cross instead of green circle? If so, click on that commit to look at the “diff” (where it shows which lines have been modified). The error is likely to have been introduced by these changes.
- Ask for help: you can open an issue on your repository and tag a member of The Carpentries team to ask for help finding the problem. Alternatively, you can also ask for help in The Carpentries Slack workspace. (More about communication channels for discussing lesson development coming later in this training.)
- Waiting for the GitHub Actions process that builds the website to run can be tedious, and slow down the process of troubleshooting: it can take a few minutes for the process to reach the point where it encounters the problem. It can be helpful to download the files for your lesson and try to build a version of the website on your local computer, where the build process will be much faster. The Workbench documentation provides instructions for installing the infrastructure and building a local preview of the lesson website.
Key Points
- The main pages of a lesson website (lesson episodes) are created
from individual Markdown or R Markdown files in the
episodesfolder of a lesson repository - Objectives, questions, keypoints, exercises (and solutions), and other “special” structural page elements (other than plain text) can be formatted using fenced div blocks - they are rendered visually differently in the lesson website.
- There are many types of fenced divs available to lesson developers.
Content from How we Operate
Last updated on 2024-06-26 | Edit this page
Overview
Questions
- What are the important milestones in the development of a new lesson?
- How can The Carpentries lesson development community help me complete my lesson?
Objectives
After completing this episode, participants should be able to…
- Describe the role that feedback plays in the life cycle of a lesson.
- Connect with other members of the community.
Until now, this training has focused on the internals of a lesson development project. Now it is time to consider the evolution of a lesson over time, and the context in which lesson development can take place.
The Lesson Life Cycle Revisited
At the beginning of this training, you were introduced to the concept of a lesson life cycle, which The Carpentries promotes to help developers communicate the status of an open source lesson project to instructors and learners.
Indicating Your Lesson’s Progress
The life cycle stage of a lesson is displayed as a banner on the lesson site, as a label on the lesson GitHub repository, and alongside the lesson whenever it is shown on The Carpentries websites and lesson listings.
The life cycle stage for a lesson is configured in the
config.yaml file we encountered when we were first
introduced to The Carpentries
Workbench, as a value for the life_cycle field. For new
lesson repositories, this value is already set to ‘pre-alpha’, so you
should not have to change it yet.
The config.yaml file also contains a
carpentry field, which can be used to adjust the styling
applied to a lesson website e.g. to make it look like a lesson from the
Software, Library, or Data Carpentry Lesson Programs.
For community-developed lessons, where no official relationship
exists with one of these Lesson Programs, you should keep using the
incubator setting, before potentially switching over to
another styling when the lesson moves e.g. into The Carpentries Lab.
Pathways out of Lesson Incubation
The Carpentries provides pathways for mature lessons to leave the Incubator.
- A mature lesson may join an existing lesson program, e.g. Data Carpentry, Library Carpentry, or Software Carpentry, subject to review and approval by the relevant Curriculum Advisory Committee.
- Developers can submit their lessons for open peer review in The Carpentries Lab, which hosts a growing collection of high-quality, stable lessons created by the community. Developers submitting to the Lab have the option of publishing their lesson in The Journal of Open Source Education (JOSE).
A lesson does not need to be stable to be useful to the community: lessons with alpha and beta status are already valuable resources to be taught and reused.
Pilot Workshops
Testimonial
“No lesson survives first contact with learners”.
In line with the importance we placed on evaluation of lesson content earlier in this training, the life cycle described above places considerable emphasis on the testing of lessons in pilot workshops.
For these pilot workshops to provide an effective evaluation of the lesson, it is essential to seize the opportunity they provide to collect feedback and data. For example, a pilot workshop offers a chance to answer questions like:
- How much time does it take to teach each section of the lesson?
- How much time is required for each exercise?
- What technical issues were encountered during the lesson?
- What questions did learners ask during the workshop?
- Which parts of the lesson were confusing for learners?
- Which exercises could be improved to provide more information to the instructors?
The pilot workshop notes template provides a starting point, but it takes a lot of time and effort to keep track of all this information during a pilot workshop. It can be helpful to assign this task to a member of the lesson development team, who can dedicate themselves to observing the pilot and taking notes. As soon as the pilot has finished, it is a good idea to take some time to share the notes with the other lesson authors, to reflect on and discuss the experience of teaching the lesson, and to synthesise the lesson notes and any action items that come from this debrief into specific action items to improve the lesson. These should be added as new issues on the lesson repository, to help you keep track of the work that needs doing for the next iteration.
For beta pilot workshops, where the lesson is taught by instructors who have not yet made a major contribution to its development, it is vital that instructors have the opportunity before the workshop to learn about the lesson from the authors, and to debrief with the authors afterwards so that they can share their experience and observations about how the lesson could be further improved.
Hosting and Teaching Pilot Workshops
The Carpentries community handbook includes guidance for community members who want to teach and/or host pilot workshops.
Connecting with the Lesson Developer Community
The Carpentries Incubator hosts a thriving community of lesson developers, working on lesson projects at every stage of the life cycle.
You can add your lesson project(s) to the Incubator by submitting an issue to the Incubator Proposals repository. Lesson developers working in the Incubator benefit from increased visibility for their projects and dedicated support from The Carpentries team.
Connecting with the lesson developer community can be a great way to find collaborators to contribute to or test your lesson, to stay up to date with the latest support provided to the community, and to learn from the experience of others.
Here are a community activities and channels that you might be interested in joining:
- The
lesson-devchannel on The Carpentries Slack workspace. This is a platform for the community to ask questions and make announcements about lesson development. You could also browse the other channels on the workspace for any that are relevant to the topic of your lesson.- You may also find it helpful to create a new channel on Slack for discussion of your lesson. Chat channels like this can be valuable ways for a remote team to communicate and collaborate.
-
The
incubator-developerslist on TopicBox. The Carpentries Curriculum Team uses this mailing list to make relevant announcements to the community of lesson developers working in Incubator. - The Carpentries Curriculum Team hosts monthly Lesson Development Coworking Sessions, which are a good opportunity to engage with other lesson developers and make regular progress on your project. The sessions are listed on The Carpentries community calendar.
Exercise: join relevant channels (5 minutes)
Use this time to explore the options listed above and join/subscribe to any communication channels that you find interesting.
Key Points
- Teaching a lesson for the first time is an essential intermediate step in the lesson development process.
- The Carpentries lesson developer community shares their experience on multiple communication channels.
Content from Preparing to Teach
Last updated on 2024-10-17 | Edit this page
Overview
Questions
- What can I do to prepare to teach my lesson for the first time?
- How should I communicate lesson setup instructions to learners?
- What information should be recorded for instructors teaching a lesson?
- How should information be collected as part of the feedback process?
Objectives
After completing this episode, participants should be able to…
- Summarise lesson content as a teaching plan.
- Add Setup Instructions and Instructor Notes to the lesson site.
- Create a feedback collection plan.
By now you should have developed objectives, exercises/formative assessments, and examples for at least one of the episodes in your lesson. In order to prepare to teach it (for the first time in particular) you’ll need to create a teaching plan and design a method to collect feedback on your lesson.
Teaching Plan
A teaching plan outlines the structure of your teaching session, including details, estimated duration and materials needed to deliver it. For example, a teaching plan may contain the following:
- welcome, introductions and motivation for the lesson - to introduce yourself, aims and objectives of the lesson and how it will help learners
- setup - to check if everyone is ready to proceed with the lesson
- segments of teaching and exercises from the lesson and any other planned activities (e.g. group discussions, wrap-up and feedback)
- checkpoints - places where you have planned to stop to check-in with learners
- slides, figures and other visual aids needed to deliver teaching and other parts of the teaching plan
- resources/references/recommended reading
Checkpoints in a Teaching Plan
Checkpoints are especially useful at the beginning of a lesson when you may not have as many exercises but the learners need to have the data downloaded or tool opened to be able to follow along. A checkpoint could include stopping to ask what questions learners have or having them indicate if they have completed the most recent step shown by the instructor.
Perspective on Preparing a Teaching Plan
“I usually create detailed notes organised in a concept map/workflow in order to make everything consistent. I also write next to each bit of teaching the time approximately needed. For each lesson I create (and reuse when I teach again the same lesson) very detailed notes of what happens during the lesson, including teaching goals and learning objectives I mean to achieve with each bit of lesson. This rich lesson plan requires a lot of work the first time I teach a lesson, but reduces a lot the preparation time of future lessons and are super useful to other teachers.”
- Dr. Allegra Via, Carpentries Instructor and Instructor Trainer
Exercise: Prepare a teaching plan (15 minutes)
Create a bullet point list or brief notes describing what you will say and do when teaching the episodes you have been focussing on during this training.
You could create this teaching plan in a document just for yourself or write it into your Lesson Design Notes document.
Figures and other visual aids are great tools to help participants understand the content being presented and make associations between pieces of information. The Instructor View of the lesson website includes a link in the top menu to Extract All Images, providing a single page containing all of the images in the lesson and their captions: this can be useful as an alternative to a slide deck when refering to figures and other visual aids during a workshop.
Setup Instructions
If your lesson requires participants to use any software, tools or
data, it is worthwhile investing time now into creating clear download,
installation and setup instructions for participants to follow and
circulating them ahead of the workshop. This will give you more time at
the pilot for teaching the lesson as troubleshooting setup problems on
the spot can be time-consuming and stressful. Each Carpentries lesson
has a dedicated place for setup instructions within The Carpentries
Workbench located in the file learners/setup.md off the
lesson project root.
Exercise: Add Setup Instructions (10 minutes)
Add setup instructions (in the learners/setup.md file)
with a list of software/tools/data needed by participants to follow your
lesson and links on how to obtain and install them.
Rather than producing a separate page in the lesson site, the
contents of learners/setup.md will be combined with
index.md to produce the Learner View of the landing page of
your lesson.
Instructor Notes
Another standard piece of The Carpentries Workbench are the
Instructor Notes pages - intended to help you and other
instructors deliver your lesson by providing useful tips for how to best
run a workshop or explain “sticky” points in the lesson. Instructor
Notes for your lesson are located in the
instructors/instructor-notes.md file off your lesson
project root - this page is displayed in the header menu when in
“Instructor View”.
You may consider adding the following information to your lesson’s Instructor Notes:
- rationale, strengths and/or weaknesses of the lesson design
- what has been tried before and what worked/did not work
- tips for teaching the lesson/particular episodes
- challenges encountered
- troubleshooting common setup/installation/exercise issues
- parts that seem to cause trouble or most confusion for learners (i.e. where to tread carefully, spend more time explaining)
You can also add notes in a specific episode of the lesson using fenced div with tag “instructor”. Instructors can see such notes within the lesson and the “Instructor Notes” tab in the header menu of the website by changing the view in the upper right-hand corner of the lesson to “Instructor View”. These notes are hidden when in ‘Learner View’
It may initially be difficult to create Instructor Notes until you have taught the lesson yourself for the first time - but after the first pilot you will have enough feedback to draft them.
Exercise: Add Instructor Notes (5 minutes)
Add Instructor Notes (in the
instructors/instructor-notes.md file) with an initial list
of points that will help you and other instructors deliver the
lesson.
Add a single instructor note in one of your episodes using the “instructor” fenced div.
Then try out the “Instructor View” of your lesson and seeing the Instructor Notes in the header menu and the in-line hidden instructor note in your episode.
The concept of hidden curriculum describes curricular effects that influence learning but are not explicitly addressed in the lesson content - e.g. (unofficial) social and cultural norms, behaviours and values that are transferred by instructors to learners, often unconsciously. These are all the additional things that people learn about the topic from the way your lesson is taught, rather than from its official content. It is worthwhile to spend some time thinking about what learners could pick up from the way your lesson is taught, as well as from its written content. You can use Instructor Notes to guide instructors on some of the aspects of the hidden curriculum of your lesson, e.g. to encourage them to follow certain tried and tested practices.
Feedback Collection Plan
There is a lot to collect feedback on when trialling a new lesson and this is arguably the most important part of your teaching plan. It can be hard to capture all this information yourself while you are teaching, hence you will need help from your team - make sure you assign the feedback collection roles ahead of your pilot. Additionally, when you are advertising your workshop, try and make it clear to the participants you are piloting a new lesson and that they will be actively helping you refine it as well as learning new things themselves. This can help focus their attention on the feedback you need to collect from them, which you should be doing:
-
constantly and throughout the workshop
- As notes taken by an observer/co-instructor - refer to the “Pilot Workshops” episode for suggestions on things to track and the pilot workshop notes template and other suggested feedback collection mechanisms to help you gather such information.
- As “minute card” feedback from learners - minute cards are short comments from learners on what is working well and what could be improved, provided frequently (e.g. at the end of each session). Like exercises used as formative assessment, minute card feedback can be provided by learners while there is still time for instructors to act on it and adjust their approach. You may find the Virtual Minute Card template helpful for workshops run online, and the Instructor Training curriculum describes how sticky notes can be used at in-person events.
- at a designated wrap-up and feedback session at the end of the workshop. Here, you can cover any other feedback that was not addressed up to that point and any specific questions you may want to ask your audience (e.g. about the choice of topics for the lesson, difficulty level, workshop operations). You may design a special form that participants can fill in anonymously - here is a post-workshop survey template for workshops that pilot a lesson in The Carpentries Incubator. You can use it as a starting point for a post-workshop survey to be shared with your learners at the end of the pilot workshop and you can supplement it with additional questions.
Exercise: homework
The final part of this training will focus on the skills needed to collaborate effectively. Before that there will be a break, during which we would like you to complete the following three tasks:
- Teach one episode of your lesson (probably the one you have been working on in these two days). See the Lesson Trial Runs page for full details.
- After your trial run has concluded (immediately after, or when you
have reviewed any feedback you collected from learners), note down your
answers to the following questions:
- What worked?
- What did not?
- What will you do differently next time?
- What will you change in your material you taught? We will refer to these notes when we reconvene for the last episode of this training.
- Based on your experience teaching the material and the feedback you
received from your learners and helpers, make a list of issues you have
identified with the material you prepared, e.g.:
- examples that did not work as expected,
- improvements that could be made to exercises,
- parts that learners found particularly challenging,
- unexpected questions or misconceptions that came up during the trial run.
What questions do you have? (15 minutes)
The homework from this workshop includes a trial run of one of the episodes you have been developing in your lesson, to a real audience.
After reading the information provided about the trial run task and thinking about this task, what questions do you have about how you should approach teaching that trial run? Is there anything you are unsure of? What resources might help you prepare for that experience?
Key Points
- Spending time on preparing your teaching and feedback collection will make you and your participants get the most out of your workshop pilot.
- Creating clear setup instructions as part of your lesson and circulating them ahead of the pilot is time well-invested and will give you more time when teaching the lesson.
- Instructor Notes are teaching tips that you should include with your lesson to help you (a few months down the line) and other instructors, who have the relevant topic knowledge but have not been involved in the lesson design and development, deliver your lesson more successfully.
Content from Part 1 Wrap-up
Last updated on 2024-10-29 | Edit this page
Overview
Questions
- What have we learned so far?
- What needs to be done before the next part of the training?
Objectives
After completing this episode, participants should be able to…
- Summarise the lesson development process introduced in this training.
- Describe the trial run task to be completed before the next part of the training
- Identify remaining tasks that should be completed before trialling lesson content.
Recent sections of this training have focused on how to design effective assessments and write a lesson that will be engaging and motivating for learners. We provided further opportunities to apply these concepts to the development of a lesson website with The Carpentries Workbench. Next, we explored The Carpentries lesson development ecosystem and how a new lesson project can fit into that. We finished up with a discussion of the importance of teaching a new lesson and how best to prepare for that experience.
The final sessions will focus on collaboration skills, exploring how trainees can curate their repository and manage their project to foster collaboration and build community around a lesson.
Before that, there will be an extended break during which trainees should conduct a trial run of the lesson content they have been developing in the workshop so far. When the training recommences, there will be an opportunity for trainees to share their experiences and reflections from those trial runs, and to discuss what impact that will have on the next stages of their lesson development.
Polish a Lesson (20 minutes)
This exercise provides you with a chance to look back over everything you have sketched out for your episode(s) and the lesson as a whole and consider what still needs to be done before it can be taught.
You can use this time however you judge will be most beneficial to your preparations for teaching your episode in a trial run.
If you are not sure how to start, consider the following prompts:
- Which of the exercises set so far in this training did you not have time to complete?
- What do you still need to add/work on?
- What can you remove/consider removing?
- How will the narrative and example data you have chosen for your lesson support teaching and assessment?
- What diagram or other visual aids could you create/add to supplement your text?
Please spend some time reflecting on what you learned so far and then provide another round of feedback to your trainers.
Organise Your Knowledge (10-15 min)
Take some time to think back on what has been covered so far, then make some notes on the most important points and actions you want to take away from that. The Trainers and other participants will not look at this - it is only for you.
If you do not know where to start, consider the following list for a starting point:
- draw a concept map, connecting the material
- draw pictures or a comic depicting one or more of the concepts
- write an outline of the topics we covered
- write a paragraph or “journal” entry about your experience of the training today
- write down one thing that struck you the most
Optional Homework: Assessing Your Lesson Design
(This exercise adapted from Via et al. 2020.)
When the first draft of your lesson content is nearly complete, consider mapping out the relationships between the objectives of your episode and the examples and exercises via which they will be taught and assessed. For example,
The read CSV and inspect demo supports Objective 2 (load a simple CSV data set with Pandas) and will be delivered using participatory live coding. The objective will be assessed with an exercise that requires learners to apply the read_csv function to another file and count the rows in the resulting DataFrame object.
- Does any of your planned content not support any learning objectives?
- Is there at least one piece of content planned for each learning objective?
- Is there a formative assessment planned for each learning objective?
Feedback (5 min)
The Trainer(s) will ask for feedback on the training so far. Take some time to provide this feedback, before moving onto the second part of this task.
Please stay in touch as you plan for your trial run.
Key Points
- We have learned about the importance of exercises and data in a lesson, the ecosystem The Carpentries provides for lesson development, and how to prepare to teach a lesson for the first time.
- Before the next part of the training, you should teach a part of your new lesson to a real audience and reflect on the experience.
Content from Reflecting on Trial Runs
Last updated on 2024-10-17 | Edit this page
Overview
Questions
- What did I learn when I taught my lesson?
Objectives
After completing this episode, participants should be able to…
- Reflect on the experience of teaching part of a lesson for the first time.
- Identify changes and improvements that can be made as a result of trialling lesson material.
Welcome back!
In part 2 of the training, we will learn more about the skills and tools you can use to become an effective collaborator on an open source lesson. Before moving on to cover that, we will spend some time reflecting on and discussing the experience of teaching new lesson material for the first time.
Exercise: Check in (10 minutes)
Re-introduce yourself and remind the group about the lesson you have been working on.
In the break since last time, you should have trialled part of the lesson you set out to develop in this training. Congratulations on organising and delivering your first pilot! We will use this session to reflect on that experience, and take some steps to ensure that the feedback and notes you gathered during the trial run are used to improve the quality of your lesson.
Discussion: trial runs (35 minutes)
Look back at the notes you took and feedback you received during your trial run (5 mins).
Now report out to the other participants about that trial run. Try to answer some or all of the following questions:
- What worked well both in terms of content and delivery?
- What did not work as well?
- How close were your time estimates to the actual time needed to teach the material and finish exercises?
- How did the audience perceive the difficulty level of the material?
- What will you do differently next time?
- What will you change in the material you taught?
- What will you change in the way you collect feedback in future pilots?
Moving forward from your first pilot, the next steps to consider are:
- Record all the identified changes and improvements that can be made
to the lesson material and assign a person from the team responsible for
each task. The above exercise provided some reflection points after a
trial run; in terms of lesson content, they may translate into the
following considerations:
- is the material too dense, does it need extra explanations?
- would adding a diagram help explain or curate things better?
- is there too much content for one episode, and do you need to split it into smaller teaching units?
- do you need to re-organise and move some content around to improve the flow and narrative?
- are there enough exercises and practical work?
- do you need to realign your lesson objectives and key messages?
- Decide if for some of the feedback you will not take action. Remember, you do not need to respond to every piece of feedback you receive. For example, it is easy to fall down the rabbit hole of adding extra material just because someone suggested it may be a useful thing - sometimes, adding a link or a spoiler pointing to extra reading is sufficient. You need to be able to draw a line under any extra modification suggestions to keep you in scope and on schedule.
- Schedule follow-up co-working sessions with your team to carry on working on fixing issues and adding new content to maintain the momentum.
- Add/Update your lessons’ Instructor Notes based on what you learned and to help other instructors who will teach your lesson in the future.
- Think about the timeline for your next pilot(s), even provisionally, to help you set milestones and targets to work towards.
- Consider writing up your pilot experience in a mini blog post that you can share with the community.
Designing and developing quality lessons is hard - there are many pieces of a puzzle that have to come into place: both pedagogical and organisational. Do not be disheartened with the amount and type of feedback you may have received. Even Carpentries lessons that have been around for 10+ years receive improvement suggestions and fixes almost daily. From our experience, bigger lessons that are delivered over a few days require several full pilots before they can even be considered for a beta release. Planning smaller lesson trials (where you test only a portion of a lesson) and doing them more often with a friendly audience from your local groups and close colleagues is more manageable and will help you make steady progress.
Key Points
- “Perfect is the enemy of good” - your lesson does not need to be perfect before you pilot or release it for community review. Early feedback from the target audience will help you avoid straying off your lesson plan.
- Identify changes and improvements you want to make as a result of trialling your lesson and schedule co-working sessions to work on these tasks.
Content from Collaborating with Your Team
Last updated on 2024-10-25 | Edit this page
Overview
Questions
- How can I use GitHub to manage (report, assign, track, prioritise) tasks of a lesson development project?
- What communication channels are effective for a lesson development team?
- How can I use GitHub to manage contributions from collaborators?
Objectives
After completing this episode, participants should be able to…
- Report and track issues with their lesson using the GitHub interface.
- Review and provide feedback on contributions from collaborators using the GitHub interface.
In this episode we expand on how to use GitHub effectively among a group of known collaborators, currently your lesson development group, on an open source lesson development project, setting the stage for introducing tools and practices for working with newcomers. Here, we explore GitHub features to help you keep track of what needs doing on the lesson, manage contributions and prioritise work. Making decisions and managing your project as a whole will be covered in a follow-up episode on project management. Collaboration skills covered in this and follow-up episodes are transferable to any similar open source project you may be involved with.
Managing Issues
Issues are GitHub’s framework for managing issue/bug reports, feature requests, and lists of future work. They provide a single shared record of all the problems people have found with the lesson, and improvements that could be made, along with solutions and discussions around them. This helps the team to keep track of what they need to work on later, and reduces the chance of receiving redundant reports of issues you already know about.
When you create an issue, you can add a range of details to it. An issue can be assigned to a specific team member - this can be a helpful way to know who is currently working to fix an issue or a way to assign responsibility. You can assign one or more labels to issues to classify or group them. Labels can also be given colours, allowing you to see at a glance from the Issues page the state of your project.
Lesson repositories created from the template are configured with The Carpentries standard set of labels, which are designed to help lesson developers and maintainers to indicate the type and current status of issues. Some of the most frequently used labels include:
-
type:bug- indicates something that is broken in the lesson, e.g. a factual error or broken link -
type:enhancement- indicates that the issue is proposing a way for the lesson to be improved -
type:accessibility- indicates that the issue relates to the accessibility of the lesson -
status:wait- indicates that the issue cannot or will not be worked on until something else is resolved, e.g. the solution to the issue is dependent on another change yet to be made (see also:status:blocked) -
status:waiting for response- indicates that further progress cannot be made until someone (usually the person who submitted the issue) has responded -
status:in progress- indicates that someone is working on a resolution for the issue, preventing potentially redundant efforts from being made by others -
good first issue- indicates that the issue requires relatively little knowledge of the project to fix, making it a good choice for first-time contributors -
help wanted- indicates that a maintainer wants help on an issue or pull request
GitHub provides a default set of labels for repositories which can be used “as is” or customised (as done by The Carpentries above) - check the GitHub documentation on Issues for the full reference. You can also create your own custom labels to help with classifying issues.
If you are developing your lesson in The Carpentries Incubator,
you can register the repository for inclusion in The Carpentries Help Wanted Issues
page, which will list all issues labeled with
help wanted and/or good first issue. Issues
listed on this page are more visible to the wider Carpentries community,
making it a good way to attract spontaneous contributions from
newcomers.
Exercise: creating issues (15 minutes)
After your trial run, you should have created a list of issues you identified with your lesson. Open issues on the GitHub repository for your lesson, for each item on that list. Remember to add labels to each issue according to its type, priority, etc. As you create them, you may also want to assign these issues to a member of your lesson development team.
Managing Contributions
Although it is possible (for collaborators with sufficient access to
the project) to make changes to a lesson directly on the
main branch of the repository, doing so is not
recommended:
- It increases the likelihood of unintentional conflicts being created when two collaborators work on the same file simultaneously.
- It does not provide an opportunity for changes to be checked and discussed before they are included in the lesson. This makes it much more likely that mistakes will be included in the lesson, and that the commit history of the project will include many changes made only to fix/polish/undo previous commits.
- It prevents your team from benefiting from the powerful features that Git and GitHub provide to support and foster collaboration.
Instead, it is recommended practice to do most work in
branches: sets of changes that build on the work in the
main branch (or another) but are not included in that
branch until they are explicitly merged into it. GitHub
provides a way for this merging process to be managed, by giving a
platform for the changes to be described, discussed and reviewed, before
a decision is made to integrate or not integrate the changes in the
default branch of the project and the associated lesson. On GitHub,
these units of discussion, review, and acceptance/rejection of the
changes within a branch are called Pull Requests.
The term Pull Request originates from the early days of GitHub, when a contributor would be requesting that the project maintainer(s) use Git to fetch (download) their branch and merge the changes into the main branch of the project. In the terminology of Git, this combination of fetching and merging a set of changes is called pulling. The name Pull Request is slightly anachronistic now, because a project maintainer can often manage the whole process of reviewing a contribution in GitHub’s web interface.
Note: A common nickname for a Pull Request is PR or PRs for Pull Requests.
Submitting and Reviewing Pull Requests on GitHub
The Trainers will collaborate on a demonstration of the process laid out below.
Pull Request Activity
Note: This activity is designed to be carried out by two Trainers acting in two different roles: “(S)” denotes the Pull Request Submitter role, “(R)” the Pull Request Reviewer.
- (S) Explore list of issues on repository, choose issue to fix, make note of issue number.
- (S) Edit a file to fix the issue, commit changes to a new branch with an appropriate name. Ensure that the changes include some typo etc that the Reviewer will be able to comment on and/or suggest a fix/improvement to.
- (S) Show how to open a pull request from that new branch to
main. - (S) In Pull Request description, reference the number of the issue being fixed, using a keyword “Fixes #X” or “Resolves #X” (where X is the issue number) to ensure automatic closure.
- (S) Demonstrate how to mark the PR as a draft, and then how to mark as it as ready for review.
- (S) Request a review from the Pull Request Reviewer.
- (R) Open the new Pull Request in GitHub.
- (R) Explore the diff of the PR, by opening the Files changed tab and talking through the display.
- (R) Comment on a line in the diff.
- (R) Suggest a change to the diff, using the “Add suggestion” button
or three backticks marked
suggestion. - (R) Complete your review by adding some summary text and selecting “Request changes”.
- (R) Return to Conversation tab, to show how the PR thread looks after the review has been submitted.
- (S) Navigate to PR and the Files changed tab.
- (S) Respond to Reviewer’s comment(s), accept their suggested changes (draw attention to the option to batch multiple suggestions into a single commit).
- (S) Return to the Conversation tab and request a new review from the Reviewer.
- (R) Look again at the PR diff (Files changed) and ensure that requested changes have been made.
- (R) Approve the PR.
- (R) Thank the Submitter for their contribution.
- (R) Merge the PR.
- (R) Navigate to issue listing to confirm that the relevant issue has been automatically closed.
Practice with Pull Requests (15 minutes)
In breakout groups, assign yourselves the role of Submitter (S) and Reviewer (R), and repeat the process described above by completing the steps below. If there are any issues open on the repository that can be solved very quickly, feel free to address those with this exercise. Otherwise, the Submitter can open a Pull Request to make simple changes: typo fixes, or some completely arbitrary changes to a file. If the changes made for the exercise are not improvements to the lesson, the Reviewer should follow all the steps but close the PR at the end instead of merging it.
- (S) Edit a file and commit the changes to a new branch.
- (S) Open a new Pull Request from this new branch to
main. If your PR will fix an open issue on your repository, include “Fixes #X” at the start of the PR description, where X is the issue number. - (S) Request a review from the Reviewer.
- (R) Review the PR, making comments and suggesting changes as appropriate. When you have finished, approve the PR or request changes, based on your review.
- (S) Respond to any comments from the Reviewer and/or, if any changes were requested, make those and re-request a review when you have finished.
- (R) When all changes have been made to your satisfaction, approve the PR, thank the Submitter for their contribution, and merge the PR (Or close it if the changes were made for practice but should not be included in the actual lesson.)
Repository administrators can enforce this workflow by
protecting the default branch of the GitHub repository:
requiring anyone without Admin access to submit changes by pull
request instead of committing changes directly to main.
The GitHub documentation provides more information about this feature and other branch protection rules.
Reviewing is a Skill
Making effective review comments in a PR is something you will get better at with practice - here is some general guidance to help with reviewing a PR from a collaborator:
- Good reviews have empathy - consider the contributor’s feelings, identify positives and try not to be judgemental.
- Aim for improved/better rather than perfect.
- Make sure your review comments are specific and actionable.
- Make it clear in the comment what you want changed as part of a PR and, ideally, provide a concrete suggestion.
- Only request changes if the content is wrong or hard to understand (remember, different does not always mean better).
- If a conversation on a PR hasn’t been resolved by one or two back-and-forth exchanges, consider scheduling a meeting to discuss further.
The Turing Way has more on reviewing contributions, and a lot of the advice in Alex Hill’s blog post, The Art of Giving and Receiving Code Reviews (Gracefully) also applies to reviewing prose.
If you need more help or practice with GitHub features, you can attend a GitHub skill-up session for Carpentries Maintainers and lesson developers which uses the following curriculum. These sessions will be periodically announced on The Carpentries calendar of events, The Carpentries Slack and mailing lists.
Managing Communication
Having an open, publicly-visible list of all the issues with your project is a helpful way of letting people know you are aware of issues and you are working on them. This can indicate to an external audience that the project is active. It also provides you and your collaborators with an “at a glance” view of the state of the project, making it easier to prioritise future work.
GitHub provides a useful notification feature for collaborative work
- mentions. The mention system notifies team members when
somebody else references them in an issue, comment or pull request - you
can use this to notify people when you want to check a detail with them,
or let them know something has been fixed or changed (much easier than
writing out all the same information again in an email!). You can use
the mention system to notify individual GitHub accounts or whole teams
(when you need to mention multiple people at once and it is not
practical to list them all individually). Typing @ in
GitHub will bring up a list of all accounts and teams linked to the
repository that can be “mentioned”. You can also mention people who
aren’t linked with the repository but they won’t be listed in the
@ drop-down list so you will need to know their GitHub
username in order to mention them.
Slack is commonly used in the Carpentries community for quick, day-to-day message exchange among community members and staff. You should join the Carpentries Slack organisation in particular to connect with other lesson developers. Once you join, you can create a channel for your lesson development project under the Carpentries public Slack workspace or if you need more complex team communications you can create your own Slack workspace for free. Note that the Carpentries Slack is an enterprise workspace so all messages and files will be retained and no messages will be lost (for free workspaces only the most recent 10,000 messages can be viewed and searched and file storage limit is 5 GB).
Collaborators can also use other platforms to discuss lesson development or receive contributions from newcomers who are not yet fluent in using GitHub’s systems of communication. The Carpentries can assist with creating a mailing list specific to the development of your lesson on their TopicBox platform for managing threaded email discussions. Also make sure to join the Incubator lesson developers mailing list on TopicBox to keep an eye on announcements and discussions relating to lesson development in general within the Carpentries community.
Recommended Reading
Here are some additional resources where you can learn more about collaborating with your team in an Open Source project:
- The Guide for Collaboration section of The Turing Way
- The Communication resources section of the Contributor Experience Handbook, especially the collection of resources on productive conflict
Key Points
- GitHub’s features Issues, Pull Requests and Mentions can all be used to transparently and effectively communicate and collaborate in a team working on a lesson development project.
- Slack and emailing lists provide communication channels outside of GitHub - for quick day-to-day messaging and more permanent discussions around lesson issues and development, respectively.
Content from Collaborating with Newcomers
Last updated on 2024-09-04 | Edit this page
Overview
Questions
- What makes a project welcoming for newcomers?
- What information should I provide about the project to potential collaborators?
- How can I ensure regular progress on lesson development?
Objectives
After completing this episode, participants should be able to…
- Adjust a lesson repository to attract potential new collaborators.
- Create and modify issue and pull request templates in GitHub.
- Identify strategies to encourage regular development and maintenance of a lesson.
You may start working on projects by yourself or with a small group of collaborators you already know, but open source projects in the Carpentries community often grow beyond that small group. Community collaboration is the cornerstone of The Carpentries - all curricula and various programmes are developed together by the community members across borders, domains and backgrounds. Most initiatives are done through community consultation and collaboration is used as a pathway to empower people, give members a sense of purpose in the community and realise shared goals.
Collaboration helps bring different perspectives to solving a problem and share the load of the required work to develop, pilot, and, later on, maintain the lesson. People drop in and out of projects for various reasons and their levels of involvement may vary over time due to other commitments - for this reason it is worth taking time from the outset to make a project welcoming for newcomers in order to achieve a steady number of people working on your project at any time. You should plan to make it easy for new collaborators to join, to set up a local workspace so that they can contribute, help them find tasks so that they know what to contribute, and make the contribution process clear so that they know how to contribute. You also want to make it easy to give people credit for their work on your project as well as for others to reuse and give credit to your project. In this episode, we cover best practices for collaborating with newcomers and tools available to help. And remember, anything you do to make it easier for newcomers to contribute will be universally useful for all collaborators.
Documenting Your Lesson
For collaborative projects, it is important to spend the time on ‘external-facing’ features, such as documentation, to make the project as welcoming and as easy to get involved in as possible for newcomers who may not have a deep grasp of the project or know all its existing problems. Such documentation will be useful to yourself and other team members as well, e.g. if you are trying to come back to the project after a break or are reusing it for a new collaboration in the future.
Your lesson documentation should contain the following information, which should be kept up to date.
Description of the Project - README File
README, README.txt or
README.md is a plain text or Markdown file located in the
project’s root directory representing the default landing or home page
for repositories in GitHub (and similar project repositories) by
convention. It represents the first piece of documentation that people
will see when visiting your lesson repository, hence it should concisely
explain what the lesson is about and who it is for, and contain links
and pointers to further information. For example, README
should include:
- lesson title
- lesson description
-
rendered version of the lesson - even though this
is typically included in the
Aboutsection of the repository, it is always useful to link to the URL where the rendered lesson is available - contact information - include up-to-date email addresses or mailing lists or other details on how to get in touch with the lesson maintainers
-
contributing information - this is an opportunity
to list what kinds of contributions are sought (and what are not) and
how to get involved in lesson development for new contributors. You can
provide more details in a separate
CONTRIBUTINGfile within the repository’s root directory and link to it fromREADME -
credits/acknowledgements - make sure to credit
those who have helped in the lesson’s development or inspired it, and/or
any resources/templates that you have reused. You can also link to a
separate
AUTHORS.mdfile within the repository’s root directory to list all the people who contributed to the lesson content (if this list starts to become too large to include in theREADMEitself) -
citation - a convention is to include the citation
information for your lesson in a separate
CITATIONfile within the repository’s root directory and link to it fromREADME.mdso others can cite the use of the lesson in their own publications and media and it can also be automatically discovered by other applications. TheseCITATIONfiles are discussed in more detail below. -
license - a short description of and a link to the
lesson’s license typically contained in a separate
LICENSE,LICENSE.txtorLICENSE.mdfile within the repository’s root directory. The lesson repository created from the Carpentries lesson template already contains a defaultLICENSE.mdfile, but you should modify this to more accurately describe how the lesson content can be re-used by others.
Helping People Cite Your Lesson
The default citation file for new lessons using The Carpentries
Workbench is CITATION.cff, in Citation
File Format (CFF). CFF is defined using YAML (which we already
encountered in the config.yaml file for the lesson website)
and is the
recommended way of storing citations in GitHub. If your lesson
repository contains a CITATION.cff file, GitHub will
automatically show the citation information in the sidebar, making it
more visible and accessible for visitors to your repository. CFF is also
recognised and supported by other platforms including Zenodo and the Zotero reference manager.
The CITATION.cff file for newly-created lessons contains
only placeholder information, which should be replaced with relevant
details for your project as soon as practical. Citation information can
also be contained in a plain text file (CITATION,
CITATION.txt) or a Markdown file
(CITATION.md). In all cases, it is up to the lesson
developers to decide what information to include in their citation
file.
Digital Object Identifiers
We recommend that you obtain a Digital
Object Identifier (DOI) for your lesson as soon as feels
appropriate, at the latest when the lesson reaches the beta phase. You
can use this DOI in the citation information (e.g. in
the identifiers field of your
CITATION.cff), to allow people to cite a particular
version of your lesson, e.g. a snapshot of the lesson when it entered
beta testing, captured as a Zenodo record.
Instructions for Contributors - CONTRIBUTING File
CONTRIBUTING, CONTRIBUTING.txt or
CONTRIBUTING.md is a text or Markdown file within the
repository’s root directory where you should provide detailed
description of the ways people can send their contributions, what kinds
of contribution will be credited and in what ways. For example, at what
point would someone be listed as an author and how you will credit
contributions that are not recorded in the commit history of the
project. The latter is particularly important for newcomers who may not
be proficient with the use of GitHub’s features for collaborative work,
such as issues, mentions, pull requests and code review, but could still
provide valuable input and contributions.
Carpentries lesson repositories already have a generic Carpentries
CONTRIBUTING.md file to get you started. You should review
it and, if need be, modify it to fit your team’s needs and way of
working. For example, you may want to expand on what contributions you
are not looking for if your lesson already contains more
material than can be covered in a typical workshop. You can also detail
all channels on which you are willing to receive contributions on, if
they are different from the defaults included in the file.
Issue and Pull Request Templates
Consider setting up issue and pull request templates to help newcomers who may not have much experience working on collaborative projects in GitHub. Such templates can provide a structure for the issue/pull request description, and/or prompt them to fill in answers to pre-set questions. Both can help contributors raise issues or submit pull requests in a way that is clear, helpful and provides enough information for maintainers to act upon (without going back and forth to extract it). GitHub provides a range of default templates, but you can also write your own.
Other Documentation
Once you have set up the basic documentation about your lesson, you may consider adding the following useful information to your documentation too:
- set-up guides for installing and rendering the lesson locally. In most cases, a link to The Carpentries Workbench documentation will be sufficient. You can also contribute to the Workbench documentation to help improve it for the community.
- how to create and modify the pages in the lesson
Exercise: preparing your repository for collaboration (15 minutes)
Spend some time doing one of the following:
- Modify the
README.mdandCONTRIBUTING.mdfiles in your repository to provide the relevant information for would-be contributors to your lesson:- the name(s) of the current developer(s), linked to their GitHub profile(s) (README)
- contact information (README, CONTRIBUTING)
- guidance on how people should contribute, and what kinds of contribution you are most interested in receiving (CONTRIBUTING)
- links to any important background information (README)
- links to any funding bodies or host institutes that are supporting the development of the lesson (README)
- Create a new issue or pull request template, or modify an existing one, to guide contributors on how best to begin collaborating with you on GitHub.
- Using the
cffinitwebtool, create aCITATION.cffwith information appropriate to your project (listing authors, including the lesson title, repository URL, etc) and overwrite the current placeholder content with the result.
Groups of collaborators taking this training together should discuss first how they will assign these tasks between them.
Collaborating on a Lesson
Boosting the Visibility and Attracting New Collaborators
In addition to having the complete documentation in the lesson repository, The Carpentries community provides a number of ways to further raise the visibility of the lesson among the broader community and encourage community members to contribute to its further development. For example:
- listing issues from the lesson repository on The Carpentries Help Wanted page
- featuring your lesson in the Incubator Lesson Spotlight
- writing a blog post about the lesson for The Carpentries Blog, and/or attending a community discussion call to promote the lesson
- advertising the lesson at various Carpentries mailing lists - e.g. general discussion, instructors, regional communities or specific curriculum lists
You should also consider including your lesson in Hacktoberfest and similar wider community open source events aimed at encouraging people to contribute to open source projects periodically throughout the year.
Issue Labelling for Newcomers
You can encourage contributions to your lesson from newcomers by
using specific labels on issues to highlight suitable opportunities to
help. Apply the good first issue or
help wanted labels to issues in your repository to indicate
that the maintainers will particularly welcome help or pull requests
fixing such issues.
Noticing When Something Happens
In addition to Mentions, GitHub implements a comprehensive notifications system to keep you up-to-date with activities in the lesson repository so you can react in a timely manner. Check out GitHub’s documentation on setting notifications on individual repository. You can choose whether to watch or unwatch all events on an individual repository, or can choose to only be notified of certain event types such as issues, pull requests, mentions, etc.
If you work on multiple projects, or the projects you follow on GitHub are particularly active, the volume of notifications you receive can quickly become overwhelming. Here are some approaches you can take to help you stay on top of things, and distinguish the high-priority tasks and important updates from the regular traffic.
Email Notifications
If you want to filter, organise, and redirect email notifications from GitHub, here are some characteristics of the messages that you make use of:
- All GitHub notifications are sent from the address
notifications@github.com. - Notification emails are sent to the address
repo-name@noreply.github.com, whererepo-nameis the name of the repository where the notification was triggered. - The email subject begins with
[org-or-user/repo-name], whereorg-or-useris the name of the organisation or the username of the user who owns the repository andrepo-nameis the name of the repository where the notification was triggered. - In addition to your email address, the cc field of the message
contains an address that describes the type of event that triggered the
notification, e.g.
author@noreply.github.comfor activity on an issue or pull request that you opened,mention@noreply.github.comfor a mention of your username, orteam-mention@noreply.github.comfor a mention of a team you are a member of, etc. - The email header (metadata) includes a
mailing-listfield with an identifier in the formrepo-name.org-or-user.github.com, which can be used to filter by the project and/or its owner.
Most email clients provide configuration for rules that can be set to redirect messages to particular folders, and/or to annotate them with a mark or flag, based on this kind of information. Here is the documentation for setting such rules in Gmail.
Notifications on GitHub.com
The alternative to using email to keep track of project activity is to manage notifications on GitHub. Adjust how you receive notifications in the Notifications section of your account settings. When logged in, you can visit https://github.com/notifications to see notifications for your account, presented as a table. Through this interface, you can:
- group notifications by repository.
- see notifications for a particular repository.
- see particular types of notification e.g. mentions, issue assignments, etc.
- create more sophisticated filters using the search bar, e.g. all mentions for all repositories in a particular organisation.
The account notification settings allow you to specify certain types of notification that should be sent by email, while everything else is collected in the web interface. Whatever strategy you choose, one of the most important habits that will help you stay on top of your projects and tasks is to remember to check these notifications frequently.
Notifications for Your Lesson Repository
As a collaborator with edit access to the repository, your
notification settings on the lesson repository should already be set to
All Activity.
You can review the level of communication you will be receiving about activities in the repository by clicking the ‘Unwatch’ button on your repository homepage: the dropdown menu will show you a range of options for the different kinds of events that you will be notified about on the repository. ‘All Activity’ is a great choice to begin with because it will ensure that you do not miss out on any changes/events.
As the time progresses, feel free to revisit this setting to adjust the level of notifications to the right level for you.
Saying “No”
Not every contribution is a good fit for a project and, especially as your lesson becomes more stable, you probably will not want to accept every suggested change. If you receive a pull request or issue that does not fit to your lesson, consider the following points to help you politely decline without demotivating the contributor from contributing to your lesson (or another open source project) again:
- Thank them for taking the time to contribute.
- Explain why the contribution does not fit into the lesson, and offer suggestions for improvement if you’re able to.
-
Link to relevant documentation, if you have it.
Notes about the design of the lesson (e.g. from this training) and any
relevant discussion threads can be very helpful here. If you notice
similar repeated requests/contributions, you might want to address them
in your documentation (e.g. in the
CONTRIBUTING.mdfile) to save yourself time in future. - Close the request.
The advice above is taken from The Carpentries Maintainer Onboarding curriculum. That resource, and the Best Practices for Maintainers guide from GitHub on which it is based, make excellent further reading as you prepare to transition from your role as an active developer into another as a responsive and responsible maintainer of your lesson.
Making Progress
The following practices have been shown to help maintain steady progress with lesson development:
- being responsive to notifications about activities and mentions
- scheduling regular co-working/sprinting sessions with team members (e.g attaching your sprint sessions to other open source community activities, which may offer goodies, rewards and prizes for participants, can provide motivation and activity spikes)
- working alongside other members of The Carpentries community at Maintainer or lesson development co-working sessions
- blocking time in your calendar for issue triage/solo material writing
- planning lesson pilots in advance to help set targets
Make sure to join the Incubator lesson developers mailing list to keep an eye on announcements and discussions relating to lesson development in the Carpentry community.
Exercise: planning to collaborate (10 minutes)
Use this time to plan how you will continue to make progress with developing your lesson, and ensure that you are ready to respond to external contributors when they arrive at your lesson. This time is yours to do with as you think is best for yourself and your project, but you might consider using it to do one or more of the following:
- schedule a lesson development co-working session with your collaborators, and/or mark the next community co-working session in your calendar.
- ensure that your the notification settings are appropriately configured for your lesson repository.
- contact The Carpentries team to ensure issues from your lesson will be included on the Help Wanted page.
Recommended Reading
Here are some additional resources where you can learn more about encouraging newcomers to get involved with your Open Source project:
- Ten simple rules for helping newcomers become contributors to open projects by Dan Sholler, Igor Steinmacher, Denae Ford, Mara Averick, Mike Hoye, and Greg Wilson
- The Contributor Experience Handbook, especially the following sections:
Key Points
- “If a project doesn’t make a good first impression, newcomers may wait a long time before giving it a second chance” - Karl Fogel, the author of “Producing Open Source Software: How to Run a Successful Free Software Project”.
- You should make an active effort to attract potential collaborators and try to make them all feel welcome and included. The Carpentries Help Wanted page and featuring in the Incubator Lesson Spotlight can boost the visibility of your lesson. Creating the appropriate documentation and using GitHub features such as labels, and issue/pull request templates will help lower the barriers for contributions to your project.
- When you design your lesson with new contributors and increased accessibility in mind, you make things better for everyone in the process.
- Scheduling regular co-working sessions, blocking time in the calendar for issue triage, and setting and being responsive to GitHub notifications will ensure regular progress on the lesson.
Content from Project Management and Governance
Last updated on 2024-08-21 | Edit this page
Overview
Questions
- How can I use GitHub to manage a lesson development project?
- What strategies exist to help collaborators make decisions and govern an open source project?
Objectives
After completing this episode, participants should be able to…
- Use GitHub features to help you with planning and managing your project.
- Prepare for a decision-making meeting between collaborators.
Project planning & management
Developing a lesson is a project and, like most projects, it consists of multiple tasks. Keeping track of the list of tasks the team has to do, progress on each, prioritising tasks for future development, sprints and releases, etc., quickly becomes a non-trivial task in itself. Without a project management tool/framework, it can be hard to keep track of what’s done, or what needs doing, and particularly difficult to convey that to others in the team or share the responsibilities.
Different tools and platforms exist to help you with project planning and management - for example, GitHub’s Milestones and Projects, Asana, Trello, Miro, even sticky notes on a white board. Different tools will work best for different teams of collaborators, but making sure you use any tool to plan and manage work on your lesson development project is better than making it up as you go along.
Project Management in GitHub
Carpentries lessons are developed in GitHub so it is convenient (but not a must) to use GitHub’s Milestones and Projects to help you keep track of different components of the project and their current status.
Milestones are a way of organising issues on your project into smaller units of work (e.g. deliverables) that have deadlines and progress of which needs to be closely tracked (e.g. release management). Projects provide a way of visualising and organising the work on a higher level and are more suitable for project management tasks that are more continuous and not time-bound. While a Milestone is associated with a single repository, a Project can span over multiple Milestones and even different repositories within an organisation in GitHub.
For a good description of differences between the two, have a look at this answer on StackOverflow.
A Milestone answers the question:
“What is remaining to finish off this task?”
A Project answers the question:
“What (tasks) are we working on at the moment?”
Milestones are good for managing time-bound, structured iterations and working in short sprints on batches of issues. Dedicated sprints seem to work well for lesson development, so we will have a look at how Milestones can help with this. We recommend you to also have a look at GitHub’s documentation on Projects and see if they are suitable for your lesson development workflow.
GitHub’s Milestones
A Milestone is a collection of issues to be worked on in a given time period. Milestones are good for tracking progress as they provide an ordered list of issues which must all be resolved for the overall task to be considered complete. Milestones work like labels - they mark and group issues that are expected to be closed at some point in time.
The Milestones page in GitHub is accessible from the
Issues or Pull requests pages - from there you
can create a new Milestone, set its description and due date, sort all
Milestones by due date and see the percentage of issues completed for
your Milestones. Within an individual Milestone’s page, you can see the
list of associated issues and prioritise them by pointing to the left of
an issue until the ‘hand’ pointer appears and then drag-and-dropping it
to a new location in the list (with higher position in the list
indicating higher priority). Back on the
Issues/Pull requests page, you can assign
issues/pull requests to a Milestone and filter your issues/pull requests
by Milestones.
Exercise: prioritising issues & planning work (15 minutes)
Use Milestones to prioritise issues and plan the first sprint and/or the first lesson release with your team.
Depending on how many issues were registered on your repository, it is possible that you will not be able to work on all the issues in the first sprint. As the development of your lesson progresses, any remaining unresolved issues can be reconsidered and prioritised for another sprint/milestone.
This process of receiving new issues and requirements, prioritisation, and working on them is naturally continuous - with the benefit that at key stages your team is repeatedly re-evaluating what is important and needs to be worked on which helps to ensure real concrete progress against project goals and requirements which may change over time.
Project Governance
In addition to managing your project on a day-to-day basis, you should consider a governance model for your lesson to describe the ground rules of participation, the roles that project participants can take on and the process for decision making within the project. You may wonder why you’d ever need a formal governance process - after all, everyone in your team is collegial, polite and hard-working. However, even among the most harmonious teams you will likely realise you need an agreed way to make decisions and resolve conflicts when your team encounters their first real disagreement1.
The moment your collaborator group exceeds 3 or 4 members (and fewer if the collaborators do not all know each other equally well at the outset) you should establish and document some kind of governance process. It will help mitigate any power imbalances which are expressed when one team member (or a group of team members) are more vocal and are able to dominate the decision-making process. It is advised to make these considerations early on, before your project grows too much and introducing structure and process into it becomes more difficult and complicated. Besides, if you do not establish these processes from the start, you are likely to discover the need to do so only when you first encounter disagreement within the team: a situation unlikely to provide perfect conditions for a discussion of governance.
Here are some aspects of governing a project that you should consider. These are borrowed from the Working in teams chapter of Research Software Engineering with Python, a book on how to work productively in small teams where everyone is welcome:
- Codes of Conduct
- meeting rules
- decision-making process
- handling conflict
- making all the above obvious to newcomers
For some other examples and inspirations of governance processes, consider the following:
- Martha’s Rules for discussing and deciding on issues in group meetings (if you cannot access the publication, see Greg Wilson’s explanation of Martha’s rules)
-
GitHub’s Minimum Viable
Governance (MVG) for establishing a lightweight governance process
into free and open source projects that are run in version control
systems.
- In MVG decisions are made through consensus of the project maintainers or determined based on the maintainers’ consideration of a number of factors in “good faith” (when explicit agreement of all maintainers cannot be reached). In reality, consensus-based governance can run you into trouble when you hit an issue where maintainers do not all agree but still want to make progress. Also, the definition of “good faith” is open to interpretation and leaves plenty of room for people, especially the powerful, to get away with acting in bad faith.
- A system based on voting and majorities may help you get around this issue. MVG is designed for a one organisation-many projects setup and is too elaborate for a single lesson project. However, it includes “boilerplate” template files for a steering committee, charter, governance, Code of Conduct documents, and language laying out a preference for consensus but with voting as a fallback, and other templates that may be helpful for those starting out and unsure about how to get on with establishing a governance process.
- This list of resources related to community governance from The Center for Scientific Collaboration and Community Engagement provides a collection of links to further reading material on this topic. It includes various guides, blog posts, and examples of community governance that may inform the discussions and decisions for your project.
Exercises: decision making process (15 minutes)
These activities are intended to help you think about the human relationships between collaborators on a project and how your interactions can influence the progress you make.
As a team read the description of project governance for this training curriculum.
Discuss with your colleagues which, if any, aspects of this would work for your team and which would not. What model of governance and decision-making might work for your team?
If you have time remaining, discuss what factors influence how much power each member of your team has in the context of the project. This could be “hard”/formalised power, e.g. project collaborators in more or less senior positions within the same workplace, and/or “soft”/informal, e.g. differences in the amount of time each collaborator has available for the project, or different levels of expertise in the subject matter of the lesson, or the lesson infrastructure and associated tools such as Git and GitHub.
Write down any questions you may have for your trainers.
(Recommended reading: the Dealing with Disruptive Behaviours resource linked in the exercise for lesson developers working alone, below, may also be helpful to ensure smoother project progress in your team.)
The Dealing with Disruptive Behaviours resource from the US National Oceanic and Atmospheric Administration (NOAA) Office for Coastal Management provides guidance for meeting facilitators. It includes examples of several common behaviours that can be disruptive to meetings (and therefore to efficient progress on a project) and advice for facilitators on how to intervene and mitigate them.
- Open the Dealing with Disruptive Behaviours resource, select each behaviour, and read the descriptions.
- Which of these behaviours do you recognise in yourself? Put another way, which “character” (“Talkative Blowfish”, “Apathetic Flounder”, etc) do you most resemble?
- Think about another project you work on where you are part of a team. Can you identify any of the disruptive behaviours in your colleagues? If so, can you find anything in the suggested interventions for these behaviours that could help you in your shared project?
Recommended Reading
Here are some additional resources to learn more about Open Source project management/governance:
- Producing Open Source Software: How to Run a Successful Free Software Project by Karl Fogel
- Program Management for Open Source Projects by Ben Cotton
- The Shared Ownership in Open Source Projects section of The Turing Way
- A guide to open source project governance models from Red Hat
Key Points
- Being intentional about task/project management for your lesson development project is better than making it up as you go along.
- Different tools and platforms exist to help you with project management - chose one that suits your team best.
- Dedicated work sprints seem to work well for lesson development and GitHub’s Milestones can be used effectively to plan and manage such sprints.
- Make sure to consider and establish a governance model for your project early on - to describe the ground rules of participation and the process for decision making within the project.
The Challenges with Applying Ownership Models Retroactively section of The Turing Way provides explores further challenges with trying to introduce governance structures to an established Open Source project.↩︎
Content from Final Wrap-up
Last updated on 2024-05-13 | Edit this page
Overview
Questions
- What steps can you take to be a good collaborator?
- How can we improve this training?
Objectives
After completing this episode, participants should be able to…
- Draw a diagram or compose a written summary of the key concepts from this training.
- Articulate constructive feedback.
We have reached the end of this training.
Looking Back
Throughout this training we promoted a collaborative approach to lesson development (as many other initiatives in The Carpentries community are conducted). Collaboration helps bring different skills and perspectives to a project and share the load of the work required to bring the project successfully past the finishing line. For lesson projects, this shared effort also extends to maintaining and updating the lesson after the initial development phase.
Exercise: What steps can you take to be a good collaborator? (20 minutes)
Spend some time reflecting on collaboration aspects we covered in this training and exploring at least two of the following lesson repositories, noting down what things you noticed in these repositories that could aid collaboration:
- Collaborative Lesson Development Training (this lesson)
- Introduction to Working with Data (Regular Expressions) official Library Carpentry lesson
- Introduction to Geospatial Raster and Vector Data with Python lesson in Carpentries Incubator
- Introduction to R for Metagenomics lesson in Carpentries Lab
Trainers will then lead a discussion on what people found and steps you could take to improve your lesson repository.
Collaboration Top Tips
While you will know the best collaboration practices for your team, here are some suggestions:
- Provide ‘external-facing’ features for your lesson (such as description, objectives, contribution and ‘how to get involved’ guides, preferred way of communication) - these will be universally useful for everyone coming to your repository (your team included).
- Find the best tools that work for your team for communication, documentation and knowledge sharing, issue & project management.
- Document and share knowledge within the team to facilitate team collaboration. Share decisions and communicate with your team synchronously and asynchronously.
- Plan your meetings to get the most out of your team (e.g. set agenda, assign meeting roles - chair, note-taker, time-keeper, etc.) and be appreciative of people’s times.
- Record and manage issues with your lesson as you develop, trial and maintain it (e.g. assign people responsible for issues, categorise and prioritise them).
- Assign tasks and roles to distribute the responsibilities - this includes assigning larger roles (project management & governance) to team members and assigning specific issues/tasks to collaborators. Progress is much more likely to be made when people know what they and others should be working on.
- Dedicated sprints where the whole team works on a set of issues towards a milestone work well for lesson development and making progress on projects in general.
- Be open and welcoming to new contributors and provide different opportunities to get involved with your project (from novice to expert collaborators). Point to upskilling guides (or run upskilling sessions if possible) to bring people up to speed.
- Manage contributions in a timely manner by noticing when things happen and responding to new issues, comments and Pull Requests (PRs). Show appreciation for everyone’s contributions.
- People bring different perspectives, expertise and skills to a collaborative project and, while they can be productive and harmoniously complement each other, having some decision making and conflict resolution process in place will make sure progress can be made when disagreements arise.
Looking Ahead
By now, you should have made good progress towards a first draft of your lesson.
Referring back to the life cycle stages for lessons described earlier in the curriculum, your lesson is still in pre-alpha but hopefully approaching alpha, when it is ready to be taught in full for the first time.
As you and your collaborators continue to iterate over the lesson - teaching it, gathering feedback, reflecting on the experience, and improving it for next time - the lesson will pass through alpha into the beta and stable stages (perhaps via peer review in The Carpentries Lab).
Your role as the original developers of the lesson, and the kind of interactions/contributions taking place on your repository, will change as you move through these stages.
| Life cycle stage | Type of contribution | Your role |
|---|---|---|
| pre-alpha | new content and whole new episodes; changes/additions to lesson objectives | active development; onboarding new collaborators; project management |
| alpha | improving existing content; adding/expanding/removing content; polishing changes/typo fixes | active development; reviewing changes; documenting feedback/actions based on pilot workshops; project management |
| beta | feedback from pilot workshops; polishing changes; expanding instructor notes; suggestions for major overhauls/reorganisations | recruiting pilot instructors; reviewing changes; guiding contributors |
| stable | feedback from workshops; polishing changes; new/updated content | reviewing changes; guiding contributors; onboarding new maintainers; stepping away from the project |
Farewell and Feedback
Thank you for following this training. We hope you found it helpful to build a foundation for your current lesson project and to prepare you to collaborate on other projects in the future.
To bring the training to a close, the Trainers will lead a number of activities to gather feedback and help you reflect on the experience.
One Up, One Down (10 min)
Provide one up, one down feedback on the entire Collaborative Lesson Development Training. Remember:
- Say only one thing, and try not to duplicate. This gets harder for those who come later!
- Trainers should try not to respond, only record responses (e.g. in the Etherpad). This is also hard, but important!
Reflection exercise (15 min)
Take this time to reflect on what you have learned in this last part of the training, and take some action to plan to successfully complete the development of your lesson after you leave. It is up to you how you use this time, but you might find some inspiration in the prompts below.
- How could you use the project management features of GitHub to prepare for the next stage of the lesson development process?
- If you have been working in a collaborative group, what steps could your group take to improve the way you work together for the remainder of the project?
- If you have been working on the lesson alone so far, what steps could you take to find new collaborators to help you with the further development of your lesson?
- Based on the experience of teaching a section of your lesson, do the objectives you wrote for your lesson still seem realistic? Can you identify anything that can be added or removed now that you have completed a trial run?
Key Points
- Documenting your lesson project, curating your lesson repository, welcoming new contributions, and taking advantage of GitHub’s project management features all make it easier for people to collaborate with you.
- Any lesson can be improved with feedback, including this one.