Content from Introduction
Last updated on 2024-02-02 | Edit this page
Overview
Questions
- What is covered in this training?
- Who are the trainers?
- Who is participating?
Objectives
After completing this episode, participants should be able to…
- Explain how trainers and participants will interact throughout the workshop.
- Summarise the main skills that will be taught in this workshop.
Note: some of the content in this course is reproduced or adapted from The Carpentries Instructor Training curriculum.
Pronouns and Names
Using correct names and pronouns (e.g. “she/her”) is important to setting a tone of respect. Learning these is hard to do quickly, so we recommend displaying it prominently during the workshop.
In an online workshop, give everyone a moment to update their display name to reflect how they would like to be addressed.
At an in-person event, we recommend supplying name tags and markers, or using plain paper to create table-displayed name placards.
Note that pronouns are personal and some participants might prefer not to share them. Do not force people to share their pronouns.
Before The Training Begins
Getting to Know Each Other
If the Trainer has chosen an icebreaker question, participate by writing your answers in the shared document for the workshop.
Code of Conduct
To make clear what is expected, everyone participating in The Carpentries activities is required to abide by our Code of Conduct. Any form of behaviour to exclude, intimidate, or cause discomfort is a violation of the Code of Conduct. In order to foster a positive and professional learning environment we encourage you to:
- Use welcoming and inclusive language
- Be respectful of different viewpoints and experiences
- Gracefully accept constructive criticism
- Focus on what is best for the community
- Show courtesy and respect towards other community members
If you believe someone is violating the Code of Conduct, we ask that you report it to The Carpentries Code of Conduct Committee by completing this form.
Today’s Trainers
To begin class, each Trainer should give a brief introduction of themselves.
(For some guidelines on introducing yourself, see the Workshop Introductions section of the Instructor Training curriculum).
Now, we would like to get to know all of you.
Our First Exercise (10 minutes)
What was the best lesson you ever followed (were taught in a class, read through online, read in a book)? Try to differentiate between what was good about the performance of the teacher/trainer and what was good about the content of the lesson itself. Take a few minutes to write down some notes about your answer, then introduce yourself to the other participants and tell them about it.
Collaborative Lesson Development Training Overview
The main objective of this training is to teach you the skills you need to design and develop an effective lesson, in collaboration with other members of the community.
During the training, we will introduce the steps you can take to design and develop a lesson to meet the needs of your target audience, and give you time to begin implementing those steps during the workshop itself. By the end of the training, you can expect to have defined an outline for your whole lesson, and begun filling in a detailed plan for some of its individual sections. The content you create in the training will exist as an open source lesson website similar to the one this training is based on. You should also know how you can continue building on the lesson and some plans for how you will collaborate on the project after the training ends.
We will focus on three main areas:
Designing a Lesson
Much of the training will discuss a process to incorporate good practices in lesson design. We will explore how defining the specific skills you wish to teach early on in the development process provides a foundation from which you can build a stronger, more impactful lesson.
Building a Lesson Website
Throughout the training, while you design and begin developing the content of your lessons, we will teach you how to incorporate this into an organised and accessible website using our lesson infrastructure.
Collaborating Effectively
We believe that lessons are much more likely to succeed, and to remain useful in the long term, if they are developed collaboratively. This training will discuss some of the ways that you can welcome new collaborators and work effectively and efficiently with those you already have.
Learning How to Teach a Lesson
This training will focus on the content - how to prepare a good lesson. More about the performance - how to deliver a lesson most effectively - is covered in The Carpentries Instructor Training.
Content from Lesson Design
Last updated on 2023-11-16 | Edit this page
Overview
Questions
- What are the recommended steps to take when developing a new lesson?
- What lesson do you want to develop during and after this workshop?
Objectives
After completing this episode, participants should be able to…
- Explain the lesson design process we will be adopting for this course.
- Summarise the lessons that participants will be working on.
A Lesson Design Process
In order to design an effective lesson, we need a structured approach with the learner in mind and clearly identified goals. Throughout this training, we will use a modified version of Nicholl’s five phase paradigm for curriculum design1. Nicholls’ paradigm describes a process, commonly referred to as backward design, where those who wish to develop a new curriculum first begin by defining exactly what their learners will be able to do after they have completed the lesson/training/course. The subsequent stages of the curriculum design process involve designing content to directly meet those stated outcomes.
- Select learning outcomes
- Choose learning experiences to help learners achieve these outcomes
- Develop content to support these experiences
- Assess learner progress towards desired outcomes
- Evaluate chosen outcomes, experiences, and content based on this assessment
The last two phases of Nicholls’ paradigm involve assessing learner progress towards the desired learning outcomes and evaluating the stated objectives and current content in light of the results of that assessment. In The Carpentries, most workshops are relatively short-format, without room for an extensive assessment after the teaching has finished (a summative assessment). To account for this, our lessons place an emphasis on formative assessment: assessment of learner progress that takes place while the teaching is still going on, to give instructors opportunities to evaluate the teaching and lesson content before the end of the workshop.
To account for this, we have adapted Nicholls’ five phases in this training, to place an emphasis on assessing learning during a workshop:
- Define desired learning outcomes
- Design assessments to determine progress towards desired outcomes
- Write content to lead learners from one of these assessments to the next
- Assess learner progress towards outcomes during teaching
- (After the break) evaluate how closely the outcomes meet the objectives
Note the cyclical nature of this process: you will complete one iteration through this cycle during this training (though probably for a limited part of your lesson, rather than the whole thing). Note also that teaching the content is an essential intermediate step in the process: the importance of feedback gathered while teaching the lesson will be a common refrain throughout this training.
Your Lessons
This training will provide many opportunities for discussion of your lessons. Providing some context now for the lessons that you will be creating will help the Trainers and other participants get involved in those discussions and give you feedback as you follow the process.
Discussion (10 minutes)
Share your answers to the following questions in the shared notes, then discuss them with the Trainers, your collaborators, and the other participants.
- What is the topic of the lesson that you plan to develop based on this training?
- Have you created training material on this topic before?
- What is motivating you to create this lesson?
Iterative Development
The Carpentries community develops open source lessons, which can always be updated and may never be finished. A lesson can undergo many iterations before it reaches a relatively stable state. To reflect this, we encourage lesson developers to indicate the status of their lesson by labelling its progress through a lesson life cycle:
Each life cycle stage indicates the level of maturity of a lesson:
- pre-alpha: a first draft of the lesson is still being constructed.
- alpha: the lesson has been/is being taught by the original authors, but has not been fully tested.
- beta: the lesson is ready to be taught by instructors who have not been significantly involved in its developed to this point.
- stable: the lesson has been extensively tested by the authors and others. It can be considered broadly complete and unlikely to undergo any drastic changes without warning.
Although your lessons will probably remain in pre-alpha throughout this training, some of the content will be equally valuable at later stages and we will also point you towards resources to help with testing the lesson and gathering feedback.
Content from Identifying Your Target Audience
Last updated on 2024-01-25 | Edit this page
Overview
Questions
- Why is it so important to think about the target audience early in the process?
- How can you ensure that your lesson reaches the right audience?
Objectives
After completing this episode, participants should be able to…
- Describe the importance of aligning lesson design with the intended audience.
- Compose a list of prior knowledge required to follow a lesson.
Target Audience
Given the limited time in a short-format training, it is vital to define the scope of the lesson, i.e. what people need to know before and what they will know after the lesson. Thinking carefully about the target audience will help you with this and defining desired learning outcomes (the first step of the lesson design). Prominently displaying a description of the target audience will also help attract people with the right motivation and relevant prior knowledge to attend your workshops.
Expertise
One of the most important things we can identify about our target audience is the level of expertise they will already have in relation to the skills taught by your lesson. In The Carpentries Instructor Training curriculum, we describe three different stages of skill acquisition: novice, competent practitioner, and expert and how it directly correlates to the complexity of mental models these different groups have about a domain/topic.
Briefly, the novice is someone who does not know what they do not know, i.e., they do not yet know what the key ideas in the domain are or how they relate, the competent practitioner has enough understanding of the domain/topic for everyday purposes, and the expert is someone who can easily handle situations that are out of the ordinary and can immediately use their prior knowledge or skills when presented with a new problem in the domain.
When designing a new lesson, it is important to think about the level of expertise that you expect learners to arrive with for two reasons:
- It helps to predict what prior knowledge and mental
model learners will have of the lesson domain when they arrive. This
can enable you to make progress quickly by
- working to help learners recall (activate) that prior knowledge1,
- building on the conceptual understanding they already have2 and,
- perhaps most importantly, giving you some idea of what misconceptions they might arrive with. It is vital that misconceptions are identified and corrected early on, before learners try to incorporate new knowledge into a broken mental model. (More on this in Stay On Target.)
- People at different stages of this process need to be taught differently. For example, novices will learn more from lessons that include worked examples and are more tutorial-like i.e. focused on a specific task, with step-by-step explanations of the process, but without a lot of extra information that is not directly relevant. However, the same approach may actually hinder learning for competent practitioners who may be distracted by a step-by-step explanation of something they already have the prior knowledge of3. For learners at this level of expertise, lessons which include activities offering learners the freedom to explore options and develop their own solutions, are likely to be more effective.
Motivation
Furthermore, your lesson will be more effective if it aligns with the motivations of the target audience. Understanding the wants and needs of your target audience, what they know already and what kinds of problems they want to solve, will help you design a lesson that learners can see the value in. It will give them the impression that taking the lesson will be worthwhile (called positive expectancies in the literature).
We will look more at strategies to establish value and build positive expectancies in the next episode.
Be Specific
It can be tempting to identify a target audience only in vague terms, for example by writing that a lesson is aimed at “PhD students” or “early career researchers”. However, taking the time to focus on real people, or imagined personae, who represent your target audience will help you take time to consider the various aspects that can influence how much someone will learn from your lesson. It will also help you notice when the assumptions you are making about your target audience are unreasonable.
Most of all, it will help you stay connected to the fact that you are not your learners: they will arrive at the lesson with different priorities, interests, and challenges than your own.
Exercise: thinking about target audience (15 minutes total for both parts)
Part 1 (all, 5 minutes): think about a member of the target audience for your lesson, and answer the following questions in the context of your lesson topic:
- What is their background?
- What do they already know how to do?
- What do they want to do with the skills they will learn from your lesson?
- What problem will your lesson help them solve?
Share your answers with your collaborators. How do they compare? If you have identified different audiences, are they compatible? Or would your time be better spent focussing on one particular audience for this lesson?
Write 1-2 diagnostic questions, for use before the lesson is taught, to help you assess whether a respondent falls within the intended audience for your lesson.
Thinking more about target audience
There is more to consider about your target audience than we could capture with only the questions listed above. In your own time, you should think more about the other considerations you might need to make when writing a lesson for your audience.
For example, what vocabulary do they use? The terms you are teaching in your lesson might have a different meaning in your learners’ domain of expertise, and it can be helpful to prepare for and try to avoid confusion arising from this clash. Furthermore, might their primary language differ from yours? If so, how might this change the way you write the lesson?
Defining Prerequisite Knowledge
A very common challenge encountered in workshops is heterogeneity of expertise among the audience. When learners arrive at a workshop with a wide range of previous experience with the topic, it is difficult for the instructors to keep everyone engaged. Those who arrive with too little relevant knowledge and experience can struggle to follow the lesson content at the pace you expect, while those who arrive with too much are likely to become bored and despondent as their expectation of learning new skills is not met.
One way to try to guard against this is to publish the description of your target audience when you advertise a workshop teaching your lesson, alongside a list summarising the skills and conceptual knowledge you expect learners to arrive with. Another is to use the information you have about your target audience to ask questions of potential learners when they apply/register to join the workshop (like the diagnostic questionnaire you may have prepared in the exercise above), and use the answers they give to filter out those who fall outside your intended audience.
While valuable, this kind of pre-assessment should be approached with caution: people are often bad at self-assessment i.e. estimating our own ability to perform a task4. We can try to mitigate for this when designing the questions for a pre-workshop survey, leaving little room for inaccurate self-assessment to confound the results. But experience suggests it is very difficult to ensure that every learner in a workshop falls within the intended audience of a lesson.
Exercise: defining prerequisite knowledge (5 minutes)
Write a list of the skills/knowledge your learners will be required to have before they can follow your lesson.
If you are struggling with this exercise because your lesson audience is novices, think about skills like touch typing, using a web browser, or interacting with a command line or graphical interface. These are skills commonly overlooked by experts and competent practitioners.
Key Points
- We recommend an iterative lesson design process that begins with identifying the target audience, before defining learning outcomes, then creating assessments, writing explanatory content, and evaluating the lesson in a workshop.
- Thinking about the target audience early in the design process helps to ensure that your lesson is built around the needs and motivations of real people.
- Use the description of your target audience to help attract people with the appropriate interests and prior knowledge to your lesson.
See chapter 1, How Does Students’ Prior Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
See chapter 2, How Does the Way Students Organize Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from Defining Lesson Objectives/Outcomes
Last updated on 2023-09-29 | Edit this page
Overview
Questions
- How can describing the things you intend to teach aid the process of writing a lesson?
- How can you be specific and realistic about what you will teach in your lesson?
- What are some of the risks associated with unrealistic or undefined expectations of a lesson?
Objectives
After completing this episode, participants should be able to…
- Explain the importance of defining specific, measurable, attainable, relevant and time-bound objectives for a lesson.
- Evaluate a written lesson objective according to these criteria.
At this stage of the training, you should have a clear idea of who the target audience is for your lesson, and what knowledge, skills, and abilities you expect them to arrive with. Now it is time to consider the additional knowledge, skills, and abilities they will have by the time they leave: these are the learning outcomes of your lesson. It can feel strange to jump from one end of the process to the other like this, but clearly defining your goals early in the lesson development process is vital. As we will see in this episode, it helps you to determine the activities, examples, etc. that are appropriate for the lesson content, and provides a scope for what should and should not be included.
Why Focus on Skills?
To ensure your audience stays motivated, and your lesson feels relevant to them, we recommend that lessons focus on teaching skills rather than tools. Lessons should be centred around what you are empowering learners to do, what will be most beneficial to them, rather than a list of functions or commands you are teaching them to use. Placing the emphasis on skills over tools will help you prioritise key concepts and consider how your lesson can have the biggest impact on the way learners do their work.
Learning Objectives
The desired outcomes (the learning objectives) of a lesson should be new skills, i.e. things that the learner can do. For the vast majority of lessons, these will be cognitive skills: things learners can do with their minds. (Lessons intended to teach other kinds of skill, such as woodwork, playing a musical instrument, or making sushi, are probably better suited to a different platform than a static website.) Cognitive skills cannot be equally easily acquired: before we can apply concepts and create something new, we must attain the ability to remember and distinguish between new concepts. Remembering and distinguishing are also abilities that are often faster to gain then applying or creating.
We must try to be realistic about how far along this scale we can move learners during a single workshop/lesson. This is one reason why the target audience is so important: if we can predict what learners will know when they arrive at the lesson, we can better define the outcomes we can expect when they leave.
Defining objectives for a lesson is essential because it allows us to focus the rest of our time on developing content that is necessary for learners to reach these goals. It will help us ensure we do not miss anything important or, conversely, include anything superfluous that could use up valuable time or distract instructors and learners.
What Does an Objective Look Like?
Objectives can be defined for a lesson as a whole - what should learners be able to do at the end of a workshop teaching this lesson? - and for individual sections within it - what should learners be able to do after following this particular part of the lesson? The objectives for the current section of this training are:
These should be read as if they were endings to a sentence beginning
“At the end of this session, learners should be able to …”
Each objective starts with a verb and describes one (and only one) skill the learner will obtain.
For objectives to be as helpful as possible, they need to be written in a way that will allow us to directly observe whether or not a learner has attained the skills we want them to. This means that the skills described by our objectives should be measurable: as a general rule, action verbs such as “explain,” “choose,” or “predict,” are more helpful than passive verbs such as “know,” “understand,” or “appreciate”, which are hard to directly assess and are often open to interpretation.
A popular aid for defining learning objectives is Bloom’s Taxonomy, which divides cognitive skills into several categories. The original taxonomy arranged these categories in a strict hierarchy, which has since been disputed. Regardless of whether these skills conform to such a hierarchy, Bloom’s Taxonomy serves as a very useful bank of action verbs for use in learning objectives.
We will see how helpful it can be to use action verbs in learning objectives when we begin talking about exercise design in the coming episodes.
SMART Objectives
To assist you in defining and writing learning objectives for your lesson, it can be helpful to turn to a popular framework for defining goals: SMART. Originally proposed to aid managers in the definition of business goals, and updated and adapted since to several other domains including education (see How to Write Well-Defined Learning Objectives for example1), the SMART acronym requires goals to be specific, measurable, attainable, relevant, and time-bound.
In the context of a lesson, SMART objectives should be:
- Specific: they should clearly describe a particular skill or ability the learner should have.
- Measurable: it should be possible to observe and ascertain when the learner has learned the skill/abilities described in the objectives.
- Attainable: the learner should realistically be able to acquire the skills or abilities in the time available in a workshop/by following the text of the lesson.
- Relevant: they should be relevant to the overall topic or domain of the lesson as a whole.
- Time-bound: they should include some timeframe within which the goal will be reached. For learning objectives, this is built into the approach described above.
Note that, for any lesson/curriculum that will be taught in a fixed amount of time, attainable and time-bound are overlapping: learning objectives for your lesson will answer the question “What will learners be able to do by the end of this lesson?” and the time available to teach the lesson, combined with the expected prior knowledge of your target audience, will determine how attainable they are.
Exercise: evaluating learning objectives (15 minutes)
Look at the example learning objectives below. Fill in the table for each objective, checking off the cells if you think an objective meets the criteria or leaving it unchecked if not. You should assume each objective is for a lesson to be taught in a two-day workshop. Note down any observations you make as you move through the list. If you have time, try to imagine the titles of lessons that would have these objectives. This part of the exercise should take 10 minutes.
At the end of this lesson, learners will be able to:
- create formatted page content with Markdown.
- program with Rust.
- fully understand GitHub Actions.
| Objective | Action verb? | Specific | Measurable | Attainable |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 |
In the last five minutes of the exercise, we will discuss as a group how each objective might be improved.
| Objective | Action verb? | Specific | Measurable | Attainable |
|---|---|---|---|---|
| 1 | ✅ | ❓ | ❓ | ✅ |
| 2 | ❓ | ❌ | ❌ | ❓ |
| 3 | ❌ | ❌ | ❌ | ❓ |
Objective 1 is the closest to what we ideally want in a lesson objective, but it illustrates how difficult it can be to make an objective truly specific. For example, a more specific and measurable version of this objective could be:
write links, headings and bold and italicised text with Markdown.
Lesson Scope
One of the major challenges of lesson design is choosing what to include in a lesson: what the main points will be, in what order they will be introduced, how much detail can be provided, and how much time can be spent on each point. Especially when writing lessons for short form training like a Carpentries workshop, difficult decisions often need to be made about what can and cannot be included. Trying to fit too much content into a lesson is counter-productive2, so it is good to avoid the temptation to cram in more content than you have time to cover properly.
Writing learning objectives is a good opportunity to begin thinking about this lesson scope, and can provide assistance when you are faced with a difficult decision about what content to cut out.
For instance, consider the order in which new skills must be acquired. Before learners can begin to acquire “higher-level” cognitive skills to perform creative and analytical tasks, they must first acquire the foundational knowledge and conceptual understanding of the domain. Furthermore, these higher-level skills take longer to acquire so, unless you can expect your target audience to arrive at the lesson with the relevant foundational knowledge and understanding, it is probably unrealistic to aim to have learners completing creative tasks before its end.
As should become clear through activities in the upcoming episodes, lessons can be broadly considered as blocks of content associated with a particular learning objective. This can be helpful when making choices about content to remove, because the task can be considered in the context of taking out whole learning objectives.
Defining Learning Objectives
We have discussed the importance of defining objectives early in the lesson design process, and looked at some examples of objectives written for other lessons. Now it is time to begin defining objectives for your own.
Here are some recommendations to help you get started:
- Aim for 3-4 objectives for every 6 hours your lesson will take to teach. (For example, this curriculum is designed to be taught in ~18 hours and has ten learning objectives.)
- These objectives are for your lesson as a whole: try to define the “end point” knowledge and skills you want learners to acquire. You can think of these as the things you would test in a fictional “final exam” your learners would be able to tackle.
- Later you will write “episode-level” objectives that should define intermediate steps towards the high-level objectives you identify for your lesson here.
Exercise: defining objectives for your lesson (20 minutes)
Write learning objectives for your lesson - what do you want learners to be able to do at the end of the workshop? When writing these lesson-level objectives, try to follow the SMART framework: make them specific, measurable, attainable, relevant, and time-bound.
Exercise: reviewing lesson objectives (15 minutes)
Swap objectives written in the previous exercise with a partner (you can also explain or show them what you wrote about your target audience, but this not essential) and review them with the following questions in mind:
- Are the objectives clear?
- Do they use “action” verbs?
- Could you directly observe whether a learner had reached this objective?
Now run the objectives through this Lesson Objective Advisor tool from the University of Manchester’s Faculty of Science and Engineering. Do the results match your assessment?
- Where do the skills described in these objectives sit on the scale?
- (optional) Are these objectives realistic, given the target audience of the lesson?
Advertising your lesson
These learning objectives, as well as the list of prerequisite knowledge you defined earlier, are very useful information to include when advertising a workshop that will teach your lesson. It will help people understand whether or not the event is a good fit for them, and manage their expectations about what they will learn if they attend.
Key Points
- Defining objectives for a lesson can help to focus your content on the most important outcomes, and outline the scope of the project.
- Following the SMART framework can help make your learning objectives as useful as possible.
- Leaving objectives unrealistic or undefined increases the risk of a lesson losing focus or spending time on activities that do not help learners gain the most important skills.
Content from Episodes
Last updated on 2024-02-23 | Edit this page
Overview
Questions
- How can the objectives for a lesson be used to break its content into sections?
Objectives
After completing this episode, participants should be able to…
- identify appropriate parts of their lesson to break into individual sections
At the end of the previous section, you defined the learning objectives for your lesson as a whole.
Rather than write the lesson as a single, long document, we recommend that you break it up into chunks like chapters in a book or episodes in a season of a TV series. This can help manage learners’ cognitive load by ensuring that you organise content into coherent, more self-contained chunks, and makes it easier for instructors to schedule regular and frequent breaks while teaching. Thinking about how content can be broken down like this early in the lesson design process helps you to consider the path learners will take to reach your defined objectives, and identify the component skills and knowledge they will need to pick up on the way.
In The Carpentries, we refer to these individual parts of a lesson as episodes, to encourage lesson developers to think about them as self-contained units (with their own narrative arc) that nevertheless contribute to the larger whole (the theme or story that runs through a full season). It also helps to think about the typical length of an episode: these chunks contain 20-60 minutes of content (teaching + exercises).
Each episode will exist as a page in the website we will build for our lessons.
Planning Your Episodes
Before we can start creating the episodes of a lesson, we need to spend some time planning out the number and order of them. The learning objectives you defined for your lesson can help with this: at the very least, it is probably a good idea to have one episode dedicated to each objective. You might also find that you can “decompose” the lesson-level objectives into more finely-grained steps that learners can take towards those end points. For example, the lesson level objective
- “create formatted page content with Markdown”
may be broken down into,
- “create bold, italic, and linked text with Markdown”
- “explain the different header levels in Markdown”
- “add images with a caption and alternative text description to a Markdown document”
Some questions you might ask yourself to help break down your lesson-level learning objectives include:
- What new knowledge and skills will learners need to acquire to be ready to fulfil the overall objectives for your lesson?
- What order should these concepts and skills be introduced in? Are some dependent on others?
- If some of these concepts and skills are complex, can they be broken down even further?
Exercise: Defining Episodes for a Lesson (25 minutes)
With your team,
- Based on the lesson-level objectives and your knowledge of the lesson topic, divide the lesson up into logical blocks (episodes), that should each take approximately 20-60 minutes to teach. Think of these logical blocks as topics that you need to cover within your lesson but do not go too deep into defining learning objectives for individual episodes - we will cover that soon.
- Next, assign responsibility for one of these episodes to each collaborator in your team. They will focus on this episode for the rest of this training, and you will teach these episodes in a trial run between parts 1 and 2 of this training.
If the length of the list of episodes you create is smaller than the number of collaborators you have on your team, try assigning two people to a single episode, with each taking responsibility for a subset of the objectives defined for that episode. If you plan more episodes than you have people in your team, do not assign more than one episode to each collaborator for now, but we strongly recommend that you assign yourselves consecutive episodes at the beginning of your lesson.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from The Carpentries Workbench
Last updated on 2023-11-16 | Edit this page
Overview
Questions
- How is a lesson site set up and published with GitHub?
- What are the essential configuration steps for a new lesson?
- How do you create and modify the pages in a lesson?
Objectives
After completing this episode, participants should be able to…
- Identify the key tools used in The Carpentries lesson infrastructure.
- Complete the fundamental setup steps for a new lesson repository.
- Edit Markdown files using the GitHub web interface.
At this stage in the training, you should have a clear idea of who the people are that you want to teach this lesson to, and what exactly the skills are that you want them to learn while they are following it. It is time to begin creating a website that will present that lesson to the world!
GitHub Pages
The source of all The Carpentries lessons is made publicly-available in repositories on GitHub. By making our repositories public like this, we encourage others to help us maintain and improve our lessons, and make it as easy as possible for them to re-use and modify our lessons for their own purposes.
GitHub provides a hosting service to open source projects such as these, allowing users to present their projects to the wider world. The repository includes a complete history of the changes made between versions of the individual files in the project, and provides many features that facilitate collaboration on projects. We will learn more about some of those collaborative features later in this training. For now, we will focus on one other important feature that GitHub provides: website hosting.
Via a system called GitHub Pages, users can build and host websites from the files present in any repository on GitHub. For many years, this has been how The Carpentries presents its lesson websites to the world.
Using The Carpentries Workbench
Carpentries lesson websites are built with The Carpentries Workbench, a toolkit that converts Markdown and RMarkdown files into the HTML that can be served by GitHub Pages. We will use it now to initialise a new lesson.
A Brief History of The Carpentries Lesson Infrastructure
In the past, our lesson sites were generated by software called Jekyll, a tool built into GitHub Pages that allows the webpage content, written in the text files of the repository, to be combined with descriptions of settings, structure, and styling, to create a website. The template used by Jekyll to structure and style lessons was initially developed in 2013/2014 by Abby Cabunoc Mayes, Greg Wilson, Jon Pipitone, and Michael Hansen for Software Carpentry. It was expanded and maintained by many members of the community over almost a decade, to also support Data Carpentry, Library Carpentry, and many other lessons.
In 2022, we adopted a new infrastructure for our lesson sites: The Carpentries Workbench. Lesson sites built on the Workbench are still hosted with GitHub Pages, but no longer use Jekyll. Instead, the lessons are built using a programming language, R, and pandoc, a software designed for converting content between file formats. The Workbench combines three R packages:
- sandpaper: converts a collection of Markdown or RMarkdown files into the structure of a lesson website.
- varnish: provides the files and folders that add styling and additional functionality to a lesson website.
- pegboard: a programmatic interface to the lesson, enabling various automated validation tasks.
For lesson developers, the Workbench makes The Carpentries lesson repositories much simpler to navigate and work with.
Creating a Lesson Repository
To get started, we first need to create a new repository for our lesson. We will use a template to do this, so that the new repository contains the basic files and folders that the Workbench needs in order to build a lesson site. There are currently two templates to choose between:
- A Markdown template
- An RMarkdown template, best suited to lessons you expect to include R source code that will generate output.
One member of each participating lesson team should choose one of these templates, following the link above and completing the configuration as follows:
- add a name for the repository
- The name should be descriptive but fairly brief, with hyphens
(
-) to separate words - the name can always be changed later, via the repository settings
- The name should be descriptive but fairly brief, with hyphens
(
- make sure the “Include all branches” box is checked to copy all
branches from the template repository
- this is important as lessons are developed in branch
mainbut rendered into websites from branchgh-pages
- this is important as lessons are developed in branch
- in the “Description” field, write the title of your lesson
- choose “Public” visibility
After pressing the Create repository button, you should be presented with a brand new lesson repository, like in the picture below.
renv/ directory.Adding collaborators
To be able to add content to the lesson, your collaborators on this project will need access to the repository. To add collaborators to the repository, navigate to Settings, then choose Collaborators from the left sidebar. Now repeat the following steps for every collaborator working with us on the project.
- Click Add people and enter the GitHub username of one of our collaborators.
- Click Add USERNAME to this repository.
Your collaborator should receive an email inviting them to join the repository. After they have accepted this invitation, they should be able to edit the repository, adding new files and modifying existing ones. Only the person who created the repository will be able to adjust the repository settings.
Repository Files
The repository contains a number of files and folders. Most of these are source files for the content of our new lesson, but a few are accompanying files primarily intended for the repository itself rather than the lesson website. These are:
CODE_OF_CONDUCT.mdCONTRIBUTING.mdLICENSE.mdREADME.md.gitignore- and the
.github/folder.
We will address all of the files later in the training. For now, we will move on to complete the basic setup of the lesson.
Configuring a Lesson Repository
Since your lesson was created from a lesson template which contains
gh-pages branch, GitHub Pages should already be generating
a rendered version of your lesson from this branch. If it isn’t, the
instructor should work with groups to activate GitHub Pages.
Which Branch and GitHub Actions
Although we configure GitHub Pages to serve the lesson website from
the gh-pages branch, the default working branch for
a lesson will be main. For the rest of this
training, you should add and edit files on main, and in
future, when you open Pull Requests to update the lesson content, these
should also be made to main. The gh-pages
branch should never be modified by anyone other than the automated
actions-user account.
To see the rendering of the gh-pages branch by the
automated actions-user, you can click the “Actions” tab from the main
repo page. This tab shows the actions triggered by each commit and if
they are in-progress or completed (sucessfully or with an error). You
can also see the details of actions in more depth by clicking on an
individual commit in this page.
On the repository home page (e.g. by clicking the name of the project near the top left of the window), click the gear wheel icon near the top right, to edit the About box. Check the box that says “Use your GitHub Pages website” to add the address of your lesson site to the About box.
After following these steps, when you navigate to the pages URL, you should see a lesson website with The Carpentries logo and “Lesson Title” in the top-left corner. You may need to wait a few minutes for the website to be generated.
config.yaml
The lesson title can be adjusted by modifying the
config.yaml file in the repository. The
config.yaml file contains several global parameters for a
lesson - to determine some of the page styling, contact details for the
lesson, etc. config.yaml is written in YAML, a structured file format of
key-value pairs in the form key: value. For example, a YAML
file of personal data might include lines such as
name: 'Mei', height_m: 1.84, and
birthdate: 1899-01-12. As well as the title of the lesson,
you can and should adjust some of the other values in
config.yaml, but you should not need to add new values or
learn a lot about YAML to be able to configure your lesson. You should
also only have to add or modify single-line string values and list
entries, and not deal with multi-line strings or other data types
(e.g. numbers, booleans, dictionaries) that YAML supports. The template
config.yaml aims to guide users through examples and
annotations on how to format the values they provide e.g. if you are
modifying a value wrapped in quotation marks in the template file, it is
safest to replace it with another value within quotation marks.
Practice withconfig.yaml(5
minutes)
Complete the configuration of your lesson by adjusting the following
fields in config.yaml. Remember that FIXME values with
quotes should be replaced with another value still in quotes and FIXMEs
without quotes should be replaced by values without quotes.
-
contact: add an email address people can contact with questions about the lesson/project. -
created: the date the lesson was created (today’s date) in YYYY-MM-DD format. -
keywords: a (short) comma-separated list of keywords for the lesson, which can help people find your lesson when searching for resources online. At a minimum, include ‘lesson’, ‘pre-alpha’, and your lesson’s (human) language. For example, ‘lesson, pre-alpha, español’. -
source: change this to the URL for your lesson repository.
We will revisit the life_cycle and
carpentry fields in config.yaml later in this
training.
.Rproj file
The repository also contains a file called FIXME.Rproj,
a project file that RStudio (a commonly-used editor for working on
lessons) can use to track some options and preferences for working on
your lesson on your local machine. This file should be renamed. You can
call this file whatever you like, but the convention is for its name to
match that of the lesson repository.
README.md
The README.md file is the “front page” of your lesson repository on GitHub, and is written in Markdown. You should use it to provide basic information about the project, for anyone who finds their way to the source files for your lesson.
We will revisit README.md later in the training with
more details on what to include in this file.
Adding Lesson Content
Now that we have completed the basic configuration of a lesson site, it is time to move on and look at where the actual lesson content will be written.
Lesson Home Page
The “home page” of a lesson is created from the index.md
file: this file should contain a brief introduction to the lesson,
designed to give visitors to the page a first impression of the lesson
and whether it is appropriate for them. The file begins with a short
header, written in the same key-value pair syntax we encountered in
config.yaml.
This header configures how the site will be built by
sandpaper, one of the components of The Carpentries
Workbench, and should be left untouched. The page content begins after
the blank line that follows this header.
Markdown Rendering on GitHub
You might have noticed by now that GitHub provides a preview of
content in Markdown files, and displays an interpreted (“rendered”)
version of Markdown file content by default in its web interface. You
might also notice that the content of index.md is displayed
differently when viewed on GitHub from how it appears on the homepage of
your lesson website: the header mentioned above appears in a tabular
layout on GitHub, but is not visible on the lesson homepage.
This is because the lesson infrastructure accepts an expanded set of elements in the source Markdown files: syntax for components of a lesson website that are not part of the standard set of Markdown features, e.g. page metadata such as the YAML header above, or blocks to make exercises and solutions visually distinctive. In these cases, you may see GitHub render the content differently from how it appears on your website, or not at all.
We will encounter more examples of this throughout the lesson, but the important thing to remember for now is that how GitHub displays source (R) Markdown files is not a reliable indicator of how content will be displayed in your website.
To get started on this home page, replace the template text with the
same title and short description you added to your
README.md.
Below those, you can add the prerequisite skills you determined
earlier for your lesson, as a bullet point list to
index.md:
Episodes
The main body of the lesson is written in episodes: the
individual chunks or sections that the lesson is separated into. The
episode pages of the lesson site will be constructed from Markdown files
in the episodes folder of the lesson repository.
The episodes folder of the new repository contains a
single file, introduction.md. The content of this file
includes examples of how to write Markdown files for The Carpentries
Workbench.
There are two important things to note:
- Like
index.md, the episode file begins with a short header. The fields in this header describe the episode title and the estimated time (in minutes) required to teach it and complete the exercises. - The example content includes several lines that start with strings
of colons (
:::::::). These mark the beginnings and ends of structural elements within the page, called fenced divs. Each fenced div starts and ends with a string of colons. A word at the end of the starting colons indicates what kind of block it is. We will explore them in more detail later in the training.
Creating a new episode
Let’s create a new episode file, for one of the episodes you have
just identified. First, open the “raw” view of the
introduction.md example episode, and copy the first 19
lines, down to the blank line under the closing string of the
objectives div.
MARKDOWN
---
title: "Using Markdown"
teaching: 10
exercises: 2
---
:::::::::::::::::::::::::::::::::::::: questions
- How do you write a lesson using Markdown and `{sandpaper}`?
::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::: objectives
- Explain how to use markdown with The Carpentries Workbench
- Demonstrate how to include pieces of code, figures, and nested challenge blocks
::::::::::::::::::::::::::::::::::::::::::::::::Now create a new file in the episodes folder. Based on
the episodes you planned out in Defining lesson objectives,
choose a name for it that concisely describes the intended content,
e.g. data-visualisation.md (or
data-visualisation.Rmd for a lesson based on R Markdown).
It is vital to include the file extension when naming this file: only
files with the .md or .Rmd extensions will be
built into webpages by the lesson infrastructure.
For page content, paste those first 19 lines of the
introduction.md file and:
- replace the title
- set the
teachingandexercisesfields to zero for now - replace the contents of the
questionsandobjectivesdivs with “- TODO”
Adding a new episode to the lesson navigation
This new episode will not yet appear in the navigation of your lesson
site. To enable this, we need to specify where the episode should appear
in the order of the lesson. That episode order is defined in the
episodes field of the config.yaml file:
Add the name of the new episode file you created to this list in
config.yaml and commit this change, for example:
After the lesson site has been rebuilt on GitHub, you should see the episode title appear under Chapters in the left sidebar navigation of your lesson site after refreshing the webpage. Clicking on that title will take you to the episode page built from the file you created. At the top of the page body, you will find the episode title and an Overview box containing a list of the questions and objectives defined for the episode. Later, we will add more content to your chosen episodes.
TODO add a labelled screenshot of a new episode page
Exercise: practice creating episodes (10 minutes)
Repeat the steps you just saw, to create another new episode file and add it to the lesson site navigation. If you know what another episode in your lesson will be about, create the page for that. Otherwise, feel free to use any values you like for the file name and episode title.
Using this approach, we can build up our lesson one episode at a time.
We have now learned everything we need to be able to use The Carpentries Workbench, and our focus can return to the process of developing a new lesson.
Sometimes, formatting errors and typos in the files of your repository can cause the process that builds your lesson website to fail.
You will likely first notice the failure to build the lesson if the lesson website is not showing the changes you made. You may also receive an email from GitHub about the build failure if that is your preferred way of receiving notifications (but it may take you some time to realise this). The good thing is that GitHub keeps your previous lesson version online until the error is fixed and a new build is completed successfully. If you are not yet familiar with the GitHub interface, it can be difficult to pinpoint the problem.
Here are some points of advice that you can follow to help find and fix the problem when you notice the build process fail.
- When you first notice that there is a problem, stop editing your lesson: any subsequent changes that you make will not be included in your lesson site until the problem is fixed, and might introduce additional issues, making it more difficult to find the original cause.
- Look at the history of commits (the link with a reversing stopwatch icon and “NN commits” at the top of the listing of files and folders in the repository homepage). Is there a place where the build has a red cross instead of green circle? If so, click on that commit to look at the “diff” (where it shows which lines have been modified). The error is likely to have been introduced by these changes.
- Ask for help: you can open an issue on your repository and tag a member of The Carpentries team to ask for help finding the problem. Alternatively, you can also ask for help in The Carpentries Slack workspace. (More about communication channels for discussing lesson development coming later in this training.)
- Waiting for the GitHub Actions process that builds the website to run can be tedious, and slow down the process of troubleshooting: it can take a few minutes for the process to reach the point where it encounters the problem. It can be helpful to download the files for your lesson and try to build a version of the website on your local computer, where the build process will be much faster. The Workbench documentation provides instructions for installing the infrastructure and building a local preview of the lesson website.
Key Points
- Lesson sites are built from source repositories with GitHub Pages.
- A new lesson repository can be created from a template maintained by
The Carpentries, and configured by adjusting the
config.yamlfile. - The main pages of a lesson website are created from Markdown or
RMarkdown files in the
episodesfolder.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from Defining Episode Objectives
Last updated on 2024-02-22 | Edit this page
Overview
Questions
- How should objectives be written for a smaller part of a whole lesson?
- How are objectives added to an episode page?
Objectives
After completing this episode, participants should be able to…
- Define learning objectives for a section of a lesson.
- Format objectives in the individual pages of a lesson website.
In Defining Lesson Objectives, you created a list of the learning objectives for your lesson as a whole. This process is helpful as a way to set the end goal and, coupled with the expected prior knowledge of the target audience you identified, describe the scope of the lesson.
We can use the same approach for the individual episodes of a lesson, defining objectives for the episode to make clear what we intend to teach in that section. Defining these objectives before writing the episode content helps us to:
- stay focused in the episode, without spending time on non-essential topics
- determine whether learners are attaining the skills we wish to teach them (we will discuss this more in the next two episodes)
- summarise the skills the learner can expect to gain by following this section of the lesson
As before, here are some recommendations to help you define these episode-level objectives:
- Aim for 2-4 objectives per episode. If you need to write more than that, consider breaking this section down into multiple episodes.
- To ensure your episode content aligns with the overall goals for the lesson, try to identify which lesson-level objective(s) your episode will contribute to.
- In the context of the previous recommendation, consider how the objectives for your episode fit as a component of the higher-level skills/understanding defined in the lesson-level objective(s).
Exercise: define objectives for your episode (30 minutes)
- Using the same approach as you did for your whole-lesson objectives, define a set of SMART objectives for your chosen episode. (15 minutes)
- Add this list of objectives to replace the
TODOin theobjectivesfenced div of your episode file (5 minutes) - Compare your list with those created by your collaborators on the
lesson:
- are there any gaps in these objectives, i.e. anything that should be covered in these episodes but is not captured in the objectives?
- are there any overlaps, i.e. anything that looks like it will be covered more than once?
- As a group, discuss how you will address any problems identified in the previous step, and edit your objectives accordingly.
After you have defined the episode objectives, you can add them to
the objectives div as follows:
Full List of Fenced Divs
The
Workbench Component Guide provides a full list of all the different
classes of fenced div that can be included using the lesson
infrastructure. The guide specifies which divs are required for an
episode (objectives, questions, and
keypoints), and which are optional.
Now that we have defined objectives for our episodes, we can start working on the next step in developing an effective lesson: designing exercises that will assess whether a learner has learned what you aimed to teach them.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from Stay on Target
Last updated on 2024-02-19 | Edit this page
Overview
Questions
- How can you measure learners’ progress towards your lesson objectives?
- Why is it important to identify misconceptions as early as possible?
- Why should we create assessments before we have written the explanatory content of our lesson?
Objectives
After completing this episode, participants should be able to…
- Describe the importance of regular assessment while a lesson is being taught.
- Design assessments to identify the misconceptions learners might have during your lesson.
As we have seen previously, defining objectives for a lesson (or a teaching episode) can help to focus your content on the most important learning outcomes and outline the scope of your lesson project. The goal of the remaining steps of lesson development is to ensure that what learners learn from following your lesson matches its defined objectives as closely as possible. To do so, you need to develop assessments to monitor progression towards your learning outcomes.
Useful reading
Useful further reading: Course design: Considerations for Trainers – a Professional Guide.
Assessments
In order to measure progress and evaluate if and what learning occurred - we use various types of assessments:
- summative assessments - used to verify whether learners achieved the stated learning objectives after instruction.
- formative assessments - used to detect changes in learner performance during instruction, to provide feedback and insight into the learners’ developing mental models of the topic taught and to identify any old or developing misconceptions.
Summative assessments sum up what learning has been achieved after training (e.g. via exams). They give valuable data about learning attainment by individuals and entire cohorts but are not used to guide further progress. They may not be as suitable for short courses, but may be necessary for those that give marks/grades or certificates of completion.
Formative assessments are applied throughout a course and with several different purposes: they provide a way to move new information from working memory to long-term memory; they can inform instructors’ decisions about how to modify instruction to better promote learning; they also inform learners about changes they may need to make to improve their learning. Ideally, they should be used often (e.g. after every 15-20 minutes of teaching), providing opportunities to instructors to change pace and refocus learners’ attention. For short courses, formative assessments are usually more valuable and easier to implement in practice than summative assessments - they need not be complex or time-consuming, just informative enough about learning for both instructors and learners.
The most effective way to test learner understanding is to do such assessments in class - they engage all learners and allow instructors to check learners’ confidence with the content and its delivery, can help you deal with any potential misunderstandings as soon as they arise, and maximise the value of workshop for everyone. Such formative assessments also help with metacognition - the awareness a learner has that they are succeeding in learning something new.
Any instructional tool that generates feedback and is used in a formative way to check for learners’ understanding can be described as “formative assessment”. For example,
- reflection at the end of a session to help process learning - e.g. asking learners to write down things they learned, things they want to know more about and any questions they still have
- concept maps and diagrams - asking learners to reflect by drawing/labeling a concept map/diagram or writing down a list of new concepts and skills they’ve learned and (optionally) how they relate to one another or connect with previous knowledge
- checking in - gauging learners’ satisfaction and understanding using agreed signals (e.g. raising different coloured post-it/sticky notes or Zoom reactions to indicate that the pace is too fast/slow, that they completed/have not completed an exercise).
- think, pair, share - learners think about an answer to a question, pair up with a classmate to discuss their answer, and then share out the consensus they came to with the class.
Many other formative assessment tools can be found in Briggs’ list of “21 ways to check for student understanding” or Edutopia’s “56 Examples of Formative Assessment”.
Exercises are one important type of formative assessment. We will now have a look into exercises that perform misconception checks and ask students diagnostic questions; we will cover a few other types of exercises that help with retaining new knowledge in one of the later episodes.
Detecting Misconceptions
Detecting and correcting misconceptions and fixing learners’ incorrect/broken mental models is as important as presenting your learners with new knowledge and correct information. Why is it important to identify misconceptions as early as possible? When mental models are broken, learning can occur slower than you might expect1. The longer a prior incorrect model is in use, and the more extensively it has to be “unlearned”, the more it can actively interfere with the incorporation of the new correct knowledge (since it will contradict the misconceptions already present in the mental model).
Example Misconceptions
Everyday Life Example
An example misconception from everyday life (borrowed
from The Carpentries Instructor Training) is, knowing that a bigger
object (more volume) pushes out more water out of a basin than a smaller
object (less volume), assuming that the similar principle will apply for
objects of different weights. Surprisingly, heavier objects of the same
volume as lighter ones will not push out more water. 
Coding Example
Another misconception, sometimes encountered by people who are
learning programming languages such as R and Python having used
spreadsheets, is that variables in programs calculated by referencing
other variables in formulas will retain the connection and be updated
automatically when the referenced variables change. Variables in R or
Python programs can use other variables in calculations
(e.g. a = b + 3), but they do not behave like cells in
spreadsheets and updating the value of variable b after the
previous assignment will not change the value of a.
Copyright / Licencing Example
Misconceptions often arise around topics of copyright and licencing of work/content (e.g. data, software, written material). Copyright allows a creator to state that they own the work they have created. This declaration is optional - even if the creator does not explicitly assert it, copyright of the work exists from the moment of creation. A licence is a legal document which sets down the terms under which the creator is releasing what they have created for others to use, modify, extend or exploit. If this information is not provided, some will not reuse the work at all while others will assume they are free to do whatever they want with the work (as there is nothing to say that they cannot). This former is the correct assumption, even if this was not the intention of the copyright owner.
Exercise: misconceptions (5 minutes)
What are the common misconceptions learners can have about the topic of your lesson? How might you identify that misconception in your learners while they follow your lesson? Share your answer in the collaborative notes document.
Hint: Try thinking about related or common tools the learners might know and how applying that prior knowledge might lead to a misconception with the topic you are teaching.
Multiple Choice Questions (MCQs)
Multiple choice question (MCQ) exercises are types of a formative assessment that can help you target anticipated misconceptions. When designed carefully, each incorrect answer in a MCQ will have diagnostic power and provide valuable insight into how a mental model is broken. For example, suppose we are teaching children multi-digit addition. An example of a well-designed MCQ (borrowed from The Carpentries Instructor Training) in this case could be:
MCQ: What is 27 + 15?
- 42
- 32
- 312
- 33
The correct answer is 42, but each of the other answers provides a valuable insight:
- they do not understand the concept of a carry and are throwing it away completely
- they understand the concept of a carry and know that they cannot just discard the carried ‘1’, but do not understand that it is actually a ten and needs to be added into the next column - they are treating each column of numbers as unconnected to its neighbours.
- they understand that they need to carry ‘1’ but are adding it to the wrong column.
Their diagnostic power means that each of the wrong choices helps an instructor figure out precisely what misconceptions learners had adopted and in which ways their mental models are broken. As a result the instructor may decide to review a particular concept or change the pace of instruction. At the same time, learners get feedback about what they have misunderstood and what they need to focus their study efforts on - we call this guided practice.
Choosing Plausible Distractors
When using a multiple choice question for formative assessment, the incorrect answers you provide as options are at least as important as the correct answer because they offer the most useful insight into the mental model your audience is building. But choosing incorrect answers can be difficult, especially early in a lesson where you are more likely to encounter misconceptions that learners have arrived with as opposed to those they have picked up while following the lesson.
Prior Knowledge and Plausible Distractors
When identifying plausible distractors, it is essential to consider the prior knowledge you expect learners to have of your lesson topic.
The plausible distractors in the MCQ example above are chosen on the assumption that learners have been taught addition by “carrying” numbers: the incorrect answers are designed to diagnose misconceptions associated with that method.
However, as pointed out by Maneesha Sane, if a learner has been taught to add numbers by another method, e.g. first rounding them up or down to values that are more easy to remember and combine (e.g. 27 to 30), they might have misconceptions that cause them to arrive at an answer that is incorrect but not represented in the MCQ options.
Example MCQs
Coding MCQ
Thinking back of the misconceptions of how values are assigned, referenced and updated in programming languages, here is an example MCQ that can probe learners for such misconceptions.
MCQ: Look at the following 3 assignment statements in Python.
What is the result at the end of the above assignments?
- n = 300 and m = 300
- n = -100 and m = -100
- n = 200 and m = 300
- n = -100 and m = 300
The correct answer is d., while plausible distractors identify different misconceptions:
- they understand the value held by
nis now also held bymbut do not understand the value ofnhas been updated since the initial assignment (e.g. treating it as a constant). - they do not understand
nandmare separate variables/containers for values (which can hold the same values) but rather assume thatmis referencing variableninstead of copying its value at the time of assignment (e.g. behaving as a reference to another cell in a spreadsheet). - they think that the reassignment for
nactually subtracts100from the original value.
Copyright / Licencing MCQ
An example MCQ to check on people’s misconceptions around licencing and reusing other people’s work could be designed as follows.
MCQ: Which of the following statements are true and which are false?
- I don’t need permission because I am only using the copyrighted work
in educational or non-profit purposes
- I should always know the licence of any code, data, libraries,
pictures or other work that you reuse or redistribute
- Since I’m planning to give credit to the authors who created the
work I reuse, I do not have to worry about or need permission
- Material I obtain from the Internet is publicly accessible so no
explicit permission is required
- The work I want to use does not have a copyright notice on it, so it’s not protected by copyright and I’m free to use it
The correct answers are as follows:
- False - you always need an explicit permission from the creator to
use their work.
- True - you should make sure that you have the permission for all the
work that you are reusing, modifying or sharing.
- False - if you give credit to a work’s owner, that only means you
are not plagiarising other people’s work and claiming it as your own,
however that does not mean that you have the permission to use it.
- False - publicly accessible work is not the same the work in the public domain. The owner explicitly must put their work in the public domain but attaching the appropriate licence to it, before you can freely reuse it.
- False - the use of copyright notice is optional as copyright exists implicitly from the moment the work is created.
To help identify plausible distractors, you can think about problems or questions from previous training events and what people struggled with, think about your own misconceptions in the past, or ask colleagues about their experiences. It is important to consider expert awareness gap, the phenomenon where experts in a topic forget what it is like not to have a good mental model of it. Observing how others learn your topic and asking colleagues to review lesson content as you design it can help mitigate this.
You should aim to create all your assessments before you have written the explanatory content of your lesson (recall Nicholl’s backward design). These assessments will guide your lesson design process by knowing exactly which knowledge you’d expect from your learners at any point in the lesson.
Exercise: designing a diagnostic exercise (20 minutes)
Create a multiple choice question (MCQ) that could be used in your lesson, to detect the misconception you identified above. As well as the correct answer, include 1-3 answer options that are not obviously incorrect (plausible distractors) and have diagnostic power i.e. each incorrect answer helps you pinpoint the exact misconception carried by the learner. Write down what misconception is being tested for each incorrect answer.
Share your MCQ in the collaborative notes document.
Exercise: reviewing formative assessments (10 minutes)
(this exercise will only work if participants have sufficient knowledge of their partner’s topic)
The Trainers will group you into pairs.
Review the MCQ designed by your partner. When providing feedback, try to answer the following questions:
- Is the question clear and easy to understand? Could the wording be improved in some way?
- Are the incorrect answers to the MCQ plausible distractors?
- Do the incorrect answers provide diagnostic power, to help an Instructor identify the misconception the learner has?
- Are there any incorrect answers missing i.e. are there other misconceptions that could be detected with this MCQ?
Share your feedback in the collaborative notes document.
Key Points
- The goal of lesson development is to ensure that the attained curriculum matches the intended curriculum as closely as possible.
- Assessments are a way to determine whether the objectives you defined for the lesson have been reached.
- Formative assessment happens during teaching and provides feedback both to an instructor and a learner - about progress and whether learning of new concepts occurred but also about any misunderstandings and misconceptions which can hinder further learning.
- It is important to detect misconceptions as early as possible and formative assessments (such as multiple choice questions) can help us with this.
See chapter 1, How Does Students’ Prior Knowledge Affect Their Learning?, of Ambrose et al. 2010.↩︎
Content from Wrap-up
Last updated on 2023-08-21 | Edit this page
Overview
Questions
- What have we learned so far?
Objectives
After completing this episode, participants should be able to…
- Identify some actions that can be taken to build a strong foundation for a lesson.
Up to now, we have focused on some good practices for designing a lesson, and provided opportunities to implement these practices. We also progressed through the first stages of building a lesson website with The Carpentries Workbench.
When we return, we will continue to build on the foundations and complete the initial development of some parts of these new lessons. We will also explore the importance of teaching as an intermediate step in the lesson development process, and discuss how best to prepare to teach new lesson content.
Feedback & questions (5 min)
The Trainer(s) will ask for feedback on the training so far. Take some time to provide this feedback, before moving onto the second part of this task.
In the shared notes document, write down any questions you have after this first part of the training, that you hope will be answered before the end.
Reflection exercise (5 min)
Take some time to think back on what has been covered so far, then make some notes on the most important points and actions you want to take away from that. The Trainers and other participants will not look at this - it is only for you.
If you do not know where to start, consider the following list for a starting point:
- draw a concept map, connecting the material
- draw pictures or a comic depicting one of the concepts covered
- write an outline of the topics we covered
- write a paragraph or “journal” entry about your experience of the training today
- write down one thing that struck you the most
Content from Designing Assessments
Last updated on 2024-02-22 | Edit this page
Overview
Questions
- Why are exercises so important in a lesson?
- What are some different types of exercises, and when should they be used?
- How should exercises be presented in a lesson website?
Objectives
After completing this episode, participants should be able to…
- Choose the format for an exercise based on the outcome it is intended to measure.
- Display exercises and their solutions in a lesson site.
Exercise Your Memory
In a simplified model of memory individuals are equipped with two types of memory: working (also called short-term) and long-term. Long-term memory essentially has unlimited storage but is slow to access, whereas working memory is quicker to access but can only hold a limited number of items at a time. For teaching, the goal is to help learners move the new things they’ve learned from working memory into long-term memory. One of the ways lesson developers can aid in this process is through exercises. In addition to providing formative assessments for instructors and check for misconceptions and broken mental models, exercises help move new skills and concepts into long-term memory by providing learners an opportunity to practice what was recently learned. Exercises should occur frequently throughout the lesson because they move items to long-term memory and free up learners’ working memory for new items.
Creating exercises builds upon the learning objectives you created earlier in the lesson design process. You can design exercises based on the actions/skills you described in your learning objectives (the learning outcomes you intend for the lesson). This will be easier if your wrote learning objectives with specific action verbs. Specific verbs can help you decide what action you want the learners to perform in the exercise. E.g. actions such as “explain” and “describe” may be better assessed by discussions and multiple choice questions, while “solve,” “construct,” “test” and other higher-level cognitive skills may be better assessed by debugging tasks, code-and-run, or use-in-a-different-context exercises.
Exercise: Exercise Types and When to Use Them (15 minutes)
The Trainers will assign you to pairs or small groups, and give each group an exercise type to focus on. Each group should assign a notetaker, to summarise your discussion at the end of the exercise.
Read about your given exercise type in the Exercise Types chapter of Teaching Tech Together by following the relevant link below.
Then, discuss the following questions together:
- What kind of skills would an exercise of this type assess? Try to identify some action verbs like those we used to write lesson objectives earlier in the workshop.
- Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners?
- Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop?
Share the major points of your discussion in the collaborative notes document.
As you discussed with your group in the last exercise, different types of learning objectives work better for novices, while others are a better fit for competent practitioners or experts.
This can be understood in terms of the types of exercises that suit the objective: exercise types that help manage cognitive load for the learner, such as fill-in-the-blanks, faded examples or Parsons problems (which all provide a lot of the guiding process/scaffolding code and allow the learner to focus on a specific concept or skill) are a good fit for a novice, to whom all elements of the topic are new. However, these kinds of exercise do not provide an opportunity for learners to develop higher-level skills, such as the ability to create whole new functions or scripts, or to extrapolate from the examples they have seen to solve a different kind of problem. Indeed, example and exercise types that are helpful to novices may even be counter-productive for learners with a greater level of expertise1.
Thus you want to choose your objectives to fit your intended audience and your exercise formats to fit your objectives.
Testimonial
“Different types of lesson objectives (LOs) are better fit for novices, while others are better fit for competent practitioners, etc. and if exercises (formative assessments) are well aligned to LOs, [they] will automatically serve the corresponding audience. Thinking in terms of LOs (What should a learner do in order to achieve this specific LO? Is this LO exactly what learners are expected to achieve by the end of this piece of instruction? etc.) is easier than thinking in terms of LOs + audience + content. LOs should be tailored to the audience, and, if this is well done, you may stop worrying about the audience. Create LOs for the specific audience and create assessments for specific LOs.”
- Dr. Allegra Via, Carpentries Instructor Trainer
Exercise: Assessing Progress Towards an Objective (30 minutes)
Using one of the exercise formats you have learned about so far, design an exercise that will require learners to perform one of the actions described in the objectives you wrote earlier, and that assesses their ability to do so.
These should be assessments of the lower-level objectives defined for individual episodes in the lesson, as opposed to the lesson-level objectives you wrote first.
Trainees working as a team can choose whether to work together on discussing and designing a single exercise to assess a single objective, or to divide their efforts and each focus on an exercise for their own episode. If you choose to take the latter approach and finish with time to spare, spend the remainder reviewing and providing feedback on one another’s assessments.
Assessing Knowledge Before a Workshop
The same approach to designing exercises within a lesson can also be used to create a short “re-assessment” questionnaire for potential learners to complete when they register for a workshop teaching the lesson (or for self-evaluation before following the lesson on their own). You can use the list of prerequisite knowledge that you defined earlier to help with this.
If you collect the results of this questionnaire, use it to follow up with people who have registered for the workshop but do not fit the intended target audience, to manage their expectations about how useful the workshop will be for them.
Demo of Writing an Exercise
Well-designed exercises are one of the most valuable resources for an instructor and any time spent on this is well invested.
To create an exercise in The Carpentries Workbench, you can use colon-delimited sections called ‘fenced divs’. In fact, there are many types of boxes in the lesson infrastucture that use fenced divs. In the Workbench, exercises are divided into two categories: discussions (where the main task is for participants to discuss a topic or prompt) and challenges (where the main task is a problem to be solved).
To start a challenge fenced div the line must contain at least 3
colons, then the challenge tag. Then the content of the
challenge is included on the following lines. Finally, you can close the
fenced div using another line with least 3 colons.
MARKDOWN
:::::::::::::::::::::::::::::::::::::: challenge
### Challenge Title
Challenge text, code, and other information goes here
::::::::::::::::::::::::::::::::::::::::::::::::::If you also want to include an expandable solution box for the
challenge you can add a solution fenced div within the challenge box.
The format is the same as for a challenge except the tag is
solution instead. Note the solutions can all be expanded
for more accessible reading using the “Expand All Solutions” option at
the top of each episode.
:::::::::::::::::::::::::::::::::::::: challenge
### Challenge Title
Challenge text, code, and other information goes here
:::::::::::::: solution
### Solution Title
Solution text, code, and other information
:::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::
For readability, you may want to have the length of the closing lines match the opening lines. See in the above how the challenge and the nested solution’s closing lines are similar lengths to the their corresponding opening lines. For more information about creating exercises see the Workbench Documentation for Exercises.
Exercise: More Practice with Fenced Divs (10 minutes)
Return to the bulleted list of prerequisite knowledge or skills you
added to the index.md file of your lesson and use fenced
divs to display it in a formatted box with the prereq
class. Note that all lesson objectives in fenced divs will be combined
into one box at the top of each episode.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from Example Data and Narrative
Last updated on 2023-12-20 | Edit this page
Overview
Questions
- Why should a lesson tell a story?
- What considerations are there when choosing an example dataset for a lesson?
- Where can I find openly-licensed, published data to use in a lesson?
Objectives
After completing this episode, participants should be able to…
- Find candidate datasets to use in a lesson.
- Evaluate the suitability of a dataset to be used in a lesson.
- Choose examples that will prepare learners for formative assessments in the lesson.
- Develop a story
Creating a Narrative
Writing your lesson as a story helps learners stay motivated and engaged. The story you create can also help learners more easily connect how the skills they are learning now could be useful after the workshop. You can enable learners to make connections between what they learn in your lesson and their own work by creating a narrative that resembles a situation the learners might encounter there.
You may find that images and figures enhance your narrative. Images are powerful communicators, conveying a lot of information with few or no words. Images can also be distracting for learners. If you choose to include images as part of the narrative of your lesson, be sure that they are consistent with the situation you are describing so that they do not increase learners’ cognitive load.
Depending on the tool you are teaching, you might also include a particular dataset as a part of the story you are weaving into your lesson. It is common for lessons to include a dataset that is used in examples and discussed throughout. This can help you maintain a narrative flow and make the lesson feel more authentic. Even if your topic doesn’t require a dataset, deciding on a consistent narrative will help create a flow between episodes and reduce cognitive load for learners.
Varying Examples
Use varied examples if you want to teach abstract concepts.
– Neil Brown
From How Your Mind Learns to Program, a lightning talk in the Never Work In Theory series
The focus of this episode is on finding one central example to be used throughout a lesson. However, where you wish for learners to develop understanding of more abstract concepts, you should try to provide a variety of examples with a common theme. This could be achieved by providing multiple opportunities to recognise and apply the concept in taught examples, activities, and exercises.
Finding Images
Copying an image from a website is technologically simple but can be legally and ethically complex. Images are intellectual property and are subject to intellectual property laws including, but not limited to, copyright and trademark laws. These laws differ by country but are consistent in theme: do not take intellectual property that does not belong to you without permission.
When looking for images that illustrate the narrative of your lesson, avoid copying images that do not include a reuse license. Assume that you cannot reuse these images unless you seek written permission from the image creator or owner. Instead, look for images that indicate that they are in the public domain or carry a permissive reuse license such as CC0 or CC-BY. Public domain images can be freely reused and adapted. Images carrying a reuse license can be used and adapted in accordance with their license terms.
If you cannot find reusable images that match your narrative, you can create your own images or seek help from others in the Carpentries community. When incorporating original images into your lesson, be sure to license these images to be compatible with the license on the rest of your lesson materials.
The guidance in this section is not a substitute for legal advice.
Compatible Licenses
Creative Commons offers a chart that identifies which CC licenses are compatible with each other for adaptation (“remix”) purposes: Creative Commons license comparison chart.
GNU offers commentary about a variety of licenses for free software; this resource may be valuable when considering a license for code: GNU: Various Licenses and Comments about Them.
Finding a Dataset
Testimonial
It was very distracting to me in a workshop, when we were given a dataset to work with but no-one explained what that data was and what it meant. It prevented me from following what the instructors were actually trying to teach me, while I tried to figure out the meaning of the numbers, columns names etc on my own.
When searching for a dataset to use in your lesson, there are a number of factors to consider. First, if possible you want the dataset to be an authentic use case for researchers in your target audience. This means balancing the size and complexity of a dataset while avoiding additional cognative load for learners. The dataset may need a certain number of observations to demonstrate some of the skills you are teaching or have a number of variables that is sufficiently similar to what learners may encounter in their work to feel authentic. At the same time, you want your learners to be able to interpret the data fairly easily during the lesson so they can focus on the skills you are teaching. This may mean you will need to subset your dataset by removing some observations (rows) or variables observed (columns).
You may also want to include and review a data dictionary in your lesson, explicitly taking the time to review the information included in the dataset. For inspiration, see the Social Sciences Data Carpentry data dictionary. An additional factor to consider when choosing a dataset to include is the license. You want to find a dataset where the data provider allows for you to freely use it. The best option is a dataset with a CC0 (Public Domain Dedication) license, as other licenses may have more ambiguity around data reuse. Lessons included in The Carpentries Incubator are encouraged to use CC0 licensed data, and may be required to do so to qualify for peer review in The Carpentries Lab. Even with a CC0 license, you will still want to follow best practice in giving attribution to the data provider or collecting agency.
More CC0 License Reading
If you’d like to read more about CC0 and CC-BY, Katie Fortney wrote an excellent blogpost on why CC-BY is not always a good fit.
An example dataset used in the Data Carpentry Ecology lessons is the Portal Project Teaching Database. This dataset is an actual ecological research project that was simplified for teaching. The reuse of this dataset throughout the Data Carpentry Ecology lessons helps stitch together the process of data analysis throughout the workshop, from data entry and cleaning to analysis and visualization.
When deciding on a dataset, it is also important to consider the ethical use of each dataset considered. Does the data contain personally identifiable information? Was the data collected without permission from the groups or individuals included? Will the data be upsetting to learners in the workshop? If the answer to any of these questions might be yes, you will need to do more research on it and continue to look for other options. Commonly data is misused from historically excluded and exploited groups. The CARE Principles for Indigenous Data Governance1 (Collective Benefit, Authority to Control, Responsibility, and Ethics) are good starting point for thinking about data sovereignty and considering the ethics of data collected about an individual or groups of people.
Examples of Public Repositories
When looking for data to reuse, consider public repositories in the subject area for your lesson or general data repositories such as:
- Dryad
- The Data Curation Network’s datasets
- The Offical Portal for European Data
- DataONE
- The Official Portal for Argentina Data (in Spanish)
- LANFRICA
People learn faster if they are motivated2, and learners will be motivated if you teach most useful things first. As you think about what story your lesson will tell, it is important to put the pieces that are most interesting to learners up front. If they are able to quickly learn tools or skills that they see as useful from your lesson, they will be more interested in continuing to learn other concepts that are needed. You may notice this trend in many of the Software Carpentry lessons. In particular, many of the coding language lessons (R and Python) have the learners create a plot very early in the lesson and then go back and teach coding fundamentals such as loops and conditionals. Visualization of data is often a very motivating and much needed skill by learners. Getting to visualization early, keeps learners interested in learning the additional and vital skills where the application might be less clear.
Exercise: Examples Before Exercises (20 minutes)
Looking back at one of the exercises you designed before: what examples could you include in your narrative to teach learners the skills they will need to apply to complete the formative assessments you have designed?
Examples: In the Software Carpentry Plotting and Programming with Python Lesson: Exercise to load and inspect CSV file for Americas -> In the lesson, the Instructor demonstrates how to load the data table for another continent (Oceania) and explores the values with a few different functions. This shows learners how to call the function to load the CSV into a data frame, and demonstrates what success looks like for this task. In the Python Interactive Data Visualization Lesson in the Incubator: Exercise to find the correct widget (a slider) for an action and modify the script to use it -> In the lesson the instructor introduces a cheatsheet and documentation for interactive widgets and then creates a dropdown widget for the application. The slider widget required in the exercise has not been demonstrated but the preceding example shows all of the necessary steps to add a widget, and provides the supporting information that learners can consult to discover how to implement the specific tool.
Outline one of these examples in your episode file.
Summary
Remember, even if you do not need a dataset for your lesson, you should decide on a narrative. Centering your lesson around a central example reduces the cognitive load of switching between examples throughout the lesson. Using an authentic, yet simple, dataset will also help reduce cognitive load and help learners to see how they might apply what they learned to their own projects. It is also important to consider licensing and ethical considerations when looking for a lesson dataset.
The evidence for this is summarised well in chapter 3, What Factors Motivate Students to Learn?, of Ambrose et al. 2010. The Carpentries Instructor Training curriculum also includes a helpful summary of how lesson content can influence Learner motivation.↩︎
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from How to Write a Lesson
Last updated on 2024-02-22 | Edit this page
Overview
Questions
- Why do we write explanatory content last?
- How can I avoid demotivating learners?
- How can I prioritise what to keep and what to cut when a lesson is too long?
Objectives
After completing this episode, participants should be able to…
- Estimate the time required to teach a lesson.
- Summarise the content of a lesson as a set of questions and key points.
- Connect the examples and exercises in a lesson with its learning objectives.
Writing Explanatory Text
Explanatory text helps connect your examples and exercises together into a cohesive lesson. If your examples and exercises are train stations and scenic views, the explanatory text is the train tracks that connect them. Often times the examples and exercises may seem like a good draft for you to teach. However, the explanatory text makes it possible for other instructors and self-directed learners to follow along by providing all the relevant information directly in the lesson. It can also help you remember your goals and stay on track teaching.
How much text to include is often a question of personal style while balancing the needs of potential users, both other instructors and learners. You want to find the balance between providing enough information for learners to meet the learning objectives without increasing cognitive load with extraneous information. In this section we will discuss factors to consider when you are writing explanatory text.
There are trade-offs to consider when preparing a site for use as an instructional guide vs use as a self-directed learning resource. Typically, self-directed learning resources are more verbose where instructional guides are aimed at an audience that can fill in the gaps on their own. However, very sparse text is less likely to be re-usable by other instructors as instructors may have different skill levels with the lesson content or differing mental models. In general, it is not a good idea to assume others, learners or instructors, will know what you were thinking when you wrote the content so, if in doubt, be explicit.
Using Callouts
Often lessons have more content than can be reasonably taught in the amount of time allotted. This is especially true for collaboratively developed lessons as each contributor/instructor may have additional items they’d like to see included, leading to “scope creep”.
To address this issue, in many Carpentries lessons, callout boxes are used for asides and short tangents, e.g. points that might be relevant to some audiences but are not essential to the flow of the lesson. These callouts should still be kept to a minimum as they can be disruptive to instructors and readers.
Lesson Time Management (10 minutes)
(5 minutes) In the shared notes document, note down your answers to these questions:
- From a design perspective, at what point is a lesson too long?
- What factors influence and constrain the length of a lesson?
- How might you prioritise what to keep if you have to cut lesson content down?
(5 minutes) In the remaining time, your Trainers will lead a discussion based on the responses.
Less is More
Trying to fit too much content into a lesson is counter-productive. It is better to cover less and provide a smaller but stable foundation for learners to build upon.
You should also consider a realistic workshop length for teaching your lesson. It is difficult to keep learners attentive for many hours over many days. Most Carpentries workshops are two work days of lesson material as it is difficult for learners to attend 4 full days of a workshop. Although, there are some in the community who will run workshops that are three days to cover additional material.
As you consider the length of your lesson discuss with your collaborators and ask yourself:
- What is essential to include?
- What can be left out if needed?
- Are there checkpoints where the lesson could end if needed?
- Can important concepts be moved up earlier to ensure they are covered?
In the end, the only way to know for sure is to teach the lesson, measuring how long it takes to teach. Borrowing from television, the Carpentries community calls these early workshops, pilot workshops. As you run pilot workshops, you can note the length of time spent in each episode. You might find the template for notes on pilot workshops helpful as it includes a table for episode and exercise timings. Instructors commonly report running short on time in workshops. Thus, it is better to assume that you are under-estimating rather than over-estimating the length. It is better to have additional time for discussion, review, and wrap-up than to rush through material to fit it into the remaining time.
If, after piloting your new lesson, you find that you did not have time to cover all the content, approach cutting down the lesson by identifying which learning objectives to remove. Then take out the objective(s) and the corresponding assessment(s) and explanatory content. You may consider making a concept map to help identify which objectives depend on one another.
Alternatively, if you decide to keep certain objectives in the lesson, you can add suggestions on which objectives can be skipped to the Instructor Notes for the lesson.
Callout
The Instructor Notes for the whole lesson should be added to the
instructors/instructor-notes.md file. This page is
displayed in the header menu when in “Instructor View”. You can also add
notes in
a specific episode of the lesson using fenced divs. Instructors can
see the notes within the lesson and the instructor notes tab by changing
the view in the upper right-hand corner of the lesson to “Instructor
View”.
Try out the instructor view by opening this episode seeing the in-line instructor note below.
We will discuss the content of Instructor Notes in more detail later on in section “Preparing to Teach”.
As you add notes and think about what to cut, remember, reducing the number of lesson objectives can help with managing learner (and instructor) cognitive load.
Testimonial
“the five ways to handle an extraneous overload situation are to:
- eliminate extraneous material (coherence principle),
- insert signals emphasizing the essential material (signaling principle),
- eliminate redundant printed text (redundancy principle),
- place printed text next to corresponding parts of graphics (spatial contiguity principle), and
- eliminate the need to hold essential material in working memory for long periods of time (temporal contiguity principle).”
Other Important Considerations for Lesson Text
When writing your lesson text you also want to avoid unintentionally demotivating your learners. Make time to review your text for:
- dismissive language - e.g. ‘simply’, ‘just’
- use of stereotypes (check learner profiles for stereotypes too)
- expert awareness gaps, i.e. places where you may be assuming the learners know more than they actually do
- fluid representations, i.e. using different terms with the same meaning interchangeably
- unexplained or unnecessary jargon/terminology (as your learners may come from different backgrounds, may be novices, not native English speakers, and a term in one domain/topic may mean something else entirely in another)
- unexplained assumptions
- sudden jumps in difficulty/complexity
You should also review your text thinking about accessibility. This includes:
- Avoiding regional/cultural references and idioms that will not translate across borders/cultures
- Avoiding contractions i.e. don’t, can’t, won’t etc.
- Checking that all figures/images have well written alternative text, including writing altnerative text for data visualizations. An alternative text contains an invisible description of an image which is read aloud to blind users by screen readers providing alternative text-based format for images. See the Workbench documentation for directions on how to add alternative text and captions to images in a lesson
- Checking the header hierarchy - no h1 headers in the lesson body, no skipped levels
- Using descriptive link text - no “click here” or “this page”, etc.
- Checking the text and foreground contrast for images
Optional Exercise: Alternative Text for Images (5 minutes)
Which of the following is a good alt-text option for the image below?
- Graph of data
- Graph with increasing lines
- Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory from 1958 to present
- Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory, Hawaii, United States, from 1959 to present including values from each year. Red line shows variation in each year and black line is average for each year. 1959 = 315.90 ppm, 1960 = 316.91, 1961 = 317.64 …
Data/Image provided by NOAA Global Monitoring Laboratory, Boulder, Colorado, USA
The third option, “Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory from 1958 to present”, is the best option. It describes the type of plot, what is measured, and the trend.
The first two options are too vague. They mention a graph but don’t give enough info to know what the graph is actually showing.
The last option is overly descriptive and starts to list the data itself. It is better to include a shorter, but informative, description and a link to the data instead.
Other Text to Complete an Episode
When writing a lesson, you should also consider adding key terms to the lesson glossary for the lesson. Following the instructions on how to create a glossary in the Workbench documentation will help you to create this section of the lesson.
Many of these terms may also be useful for other lessons and can be added to Glosario, a multilingual glossary for computing and data science terms.
In addition to objectives, a completed episode also requires fenced divs for the questions the episode answers and the keypoints a lesson covers. The questions box helps learners understand what to expect from a lesson as they might not yet understand the learning objectives. The keypoints box wraps up the lesson by providing answers to each of the questions. Keypoints also help self-directed learners review what they learned, remind instructors to recap what was covered in an episode, and help you ensure you answered all the questions you intended for an episode.
Exercise: Completing episode metadata (10 minutes)
Add key points and questions to your episode.
To check the formatting requirements, see the Introduction Episode example in your lesson or the Workbench Documentation.
Polishing a lesson
Lessons do not need to be perfected prior to teaching them the first time. In fact, keeping things relatively basic leaves you some flexibility while you run through the first few iterations of teaching the lesson, collecting feedback, and using that to guide improvements.
However, once you find that the lesson is beginning to reach something like a stable state, you may find the checklist for lesson reviewers in The Carpentries Lab useful as a guide for polishing the design and content. The checklist describes criteria for a lesson to meet a high standard in terms of its accessibility, design, content, and supporting information.
Key Points
- The objectives and assessments provide a good outline for an episode and then the text fills in the gaps to support anyone learning or teaching from the lesson.
- It is important to review your lesson for demotivating language, cognitive load, and accessibility.
- To reduce cognitive load and ensure there is enough time for for the materials, consider which lesson objectives are not needed and remove related content and assessments.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from How we Operate
Last updated on 2023-10-04 | Edit this page
Overview
Questions
- What are the important milestones in the development of a new lesson?
- How can The Carpentries lesson development community help me complete my lesson?
Objectives
After completing this episode, participants should be able to…
- Describe the role that feedback plays in the life cycle of a lesson.
- Connect with other members of the community.
Until now, this training has focused on the internals of a lesson development project. Now it is time to consider the evolution of a lesson over time, and the context in which lesson development can take place.
The Lesson Life Cycle Revisited
At the beginning of this training, you were introduced to the concept of a lesson life cycle, which The Carpentries promotes to help developers communicate the status of an open source lesson project to instructors and learners.
Indicating Your Lesson’s Progress
The life cycle stage of a lesson is displayed as a banner on the lesson site, as a label on the lesson GitHub repository, and alongside the lesson whenever it is shown on The Carpentries websites and lesson listings.
The life cycle stage for a lesson is configured in the
config.yaml file we encountered when we were first
introduced to The Carpentries
Workbench, as a value for the life_cycle field. For new
lesson repositories, this value is already set to ‘pre-alpha’, so you
should not have to change it yet.
ThecarpentryField
The config.yaml file also contains a
carpentry field, which can be used to adjust the styling
applied to a lesson website e.g. to make it look like a lesson from the
Software, Library, or Data Carpentry Lesson Programs.
For community-developed lessons, where no official relationship
exists with one of these Lesson Programs, you should keep using the
incubator setting, before potentially switching over to
another styling when the lesson moves e.g. into The Carpentries Lab.
Pathways out of Lesson Incubation
The Carpentries provides pathways for mature lessons to leave the Incubator.
- A mature lesson may join an existing lesson program, e.g. Data Carpentry, Library Carpentry, or Software Carpentry, subject to review and approval by the relevant Curriculum Advisory Committee.
- Developers can submit their lessons for open peer review in The Carpentries Lab, which hosts a growing collection of high-quality, stable lessons created by the community. Developers submitting to the Lab have the option of publishing their lesson in The Journal of Open Source Education (JOSE).
A lesson does not need to be stable to be useful to the community: lessons with alpha and beta status are already valuable resources to be taught and reused.
Pilot Workshops
In line with the importance we placed on evaluation of lesson content earlier in this training, the life cycle described above places considerable emphasis on the testing of lessons in pilot workshops.
For these pilot workshops to provide an effective evaluation of the lesson, it is essential to seize the opportunity they provide to collect feedback and data. For example, a pilot workshop offers a chance to answer questions like:
- How much time does it take to teach each section of the lesson?
- How much time is required for each exercise?
- What technical issues were encountered during the lesson?
- What questions did learners ask during the workshop?
- Which parts of the lesson were confusing for learners?
- Which exercises could be improved to provide more information to the instructors?
The pilot workshop notes template provides a starting point, but it takes a lot of time and effort to keep track of all this information during a pilot workshop. It can be helpful to assign this task to a member of the lesson development team, who can dedicate themselves to observing the pilot and taking notes. As soon as the pilot has finished, it is a good idea to take some time to share the notes with the other lesson authors, to reflect on and discuss the experience of teaching the lesson, and to synthesise the lesson notes and any action items that come from this debrief into specific action items to improve the lesson. These should be added as new issues on the lesson repository, to help you keep track of the work that needs doing for the next iteration.
For beta pilot workshops, where the lesson is taught by instructors who have not yet made a major contribution to its development, it is vital that instructors have the opportunity before the workshop to learn about the lesson from the authors, and to debrief with the authors afterwards so that they can share their experience and observations about how the lesson could be further improved.
Hosting and Teaching Pilot Workshops
The Carpentries community handbook includes guidance for community members who want to teach and/or host pilot workshops.
Connecting with the Lesson Developer Community
The Carpentries Incubator hosts a thriving community of lesson developers, working on lesson projects at every stage of the life cycle.
You can add your lesson project(s) to the Incubator by submitting an issue to the Incubator Proposals repository. Lesson developers working in the Incubator benefit from increased visibility for their projects and dedicated support from The Carpentries team.
Connecting with the lesson developer community can be a great way to find collaborators to contribute to or test your lesson, to stay up to date with the latest support provided to the community, and to learn from the experience of others.
Here are a community activities and channels that you might be interested in joining:
- The
lesson-devchannel on The Carpentries Slack workspace. This is a platform for the community to ask questions and make announcements about lesson development. You could also browse the other channels on the workspace for any that are relevant to the topic of your lesson.- You may also find it helpful to create a new channel on Slack for discussion of your lesson. Chat channels like this can be valuable ways for a remote team to communicate and collaborate.
-
The
incubator-developerslist on TopicBox. The Carpentries Curriculum Team uses this mailing list to make relevant announcements to the community of lesson developers working in Incubator. - The Carpentries Curriculum Team hosts monthly Lesson Development Coworking Sessions, which are a good opportunity to engage with other lesson developers and make regular progress on your project. The sessions are listed on The Carpentries community calendar.
Content from Preparing to Teach
Last updated on 2023-09-26 | Edit this page
Overview
Questions
- What can I do to prepare to teach my lesson for the first time?
- How should I communicate lesson setup instructions to learners?
- What information should be recorded for instructors teaching a lesson?
- How should information be collected as part of the feedback process?
Objectives
After completing this episode, participants should be able to…
- Summarise lesson content as a teaching plan.
- Add Setup Instructions and Instructor Notes to the lesson site.
- Create a feedback collection plan.
By now you should have developed objectives, exercises/formative assessments, and examples for at least one of the episodes in your lesson. In order to prepare to teach it (for the first time in particular) you’ll need to create a teaching plan and design a method to collect feedback on your lesson.
Teaching Plan
A teaching plan outlines the structure of your teaching session, including details, estimated duration and materials needed to deliver it. For example, a teaching plan may contain the following:
- welcome, introductions and motivation for the lesson - to introduce yourself, aims and objectives of the lesson and how it will help learners
- setup - to check if everyone is ready to proceed with the lesson
- segments of teaching and exercises from the lesson and any other planned activities (e.g. group discussions, wrap-up and feedback)
- checkpoints - places where you have planned to stop to check-in with learners
- slides, figures and other visual aids needed to deliver teaching and other parts of the teaching plan
- resources/references/recommended reading
Checkpoints in a Teaching Plan
Checkpoints are especially useful at the beginning of a lesson when you may not have as many exercises but the learners need to have the data downloaded or tool opened to be able to follow along. A checkpoint could include stopping to ask what questions learners have or having them indicate if they have completed the most recent step shown by the instructor.
Perspective on Preparing a Teaching Plan
“I usually create detailed notes organised in a concept map/workflow in order to make everything consistent. I also write next to each bit of teaching the time approximately needed. For each lesson I create (and reuse when I teach again the same lesson) very detailed notes of what happens during the lesson, including teaching goals and learning objectives I mean to achieve with each bit of lesson. This rich lesson plan requires a lot of work the first time I teach a lesson, but reduces a lot the preparation time of future lessons and are super useful to other teachers.”
- Dr. Allegra Via, Carpentries Instructor and Instructor Trainer
Figures and other visual aids are great tools to help participants understand the content being presented and make associations between pieces of information. The Instructor View of the lesson website includes a link in the top menu to Extract All Images, providing a single page containing all of the images in the lesson and their captions: this can be useful as an alternative to a slide deck when refering to figures and other visual aids during a workshop.
If your lesson requires participants to use any software, tools or
data - it is worthwhile investing time now into creating clear download,
installation and setup instructions for participants to follow and
circulating them ahead of the workshop. This will give you more time at
the pilot for teaching the lesson as troubleshooting setup problems on
the spot can be time-consuming and stressful. Each Carpentries lesson
has a dedicated place for setup instructions within The Carpentries
Workbench located in the file learners/setup.md off the
lesson project root.
Exercise: Add Setup Instructions (10 minutes)
Add setup instructions (in the learners/setup.md file)
with a list of software/tools/data needed by participants to follow your
lesson and links on how to obtain and install them.
Rather than producing a separate page in the lesson site, the
contents of learners/setup.md will be combined with
index.md to produce the Learner View of the landing page of
your lesson.
Another standard piece of The Carpentries Workbench are the
Instructor Notes pages - intended to help you and other
instructors deliver your lesson by providing useful tips for how to best
run a workshop or explain “sticky” points in the lesson. Instructor
Notes for your lesson are located in the
instructors/instructor-notes.md file off your lesson
project root where you may consider adding the following
information:
- rationale, strengths and/or weaknesses of the lesson design
- what has been tried before and what worked/did not work
- tips for teaching the lesson/particular episodes
- challenges encountered
- troubleshooting common setup/installation/exercise issues
- parts that seem to cause trouble or most confusion for learners (i.e. where to tread carefully, spend more time explaining)
It may initially be difficult to create Instructor Notes until you have taught the lesson yourself for the first time - but after the first pilot you will have enough feedback to draft them.
Feedback Collection Plan
There is a lot to collect feedback on when trialling a new lesson and this is arguably the most important part of your teaching plan. It can be hard to capture all this information yourself while you are teaching, hence you will need help from your team - make sure you assign the feedback collection roles ahead of your pilot. Additionally, when you are advertising your workshop, try and make it clear to the participants you are piloting a new lesson and that they will be actively helping you refine it as well as learning new things themselves. This can help focus their attention on the feedback you need to collect from them, which you should be doing:
-
constantly and throughout the workshop
- As notes taken by an observer/co-instructor - refer to the “Pilot Workshops” episode for suggestions on things to track and the pilot workshop notes template and other suggested feedback collection mechanisms to help you gather such information.
- As “minute card” feedback from learners - minute cards are short comments from learners on what is working well and what could be improved, provided frequently (e.g. at the end of each session). Like exercises used as formative assessment, minute card feedback can be provided by learners while there is still time for instructors to act on it and adjust their approach. You may find the Virtual Minute Card template helpful for workshops run online, and the Instructor Training curriculum describes how sticky notes can be used at in-person events.
- at a designated wrap-up and feedback session at the end of the workshop. Here, you can cover any other feedback that was not addressed up to that point and any specific questions you may want to ask your audience (e.g. about the choice of topics for the lesson, difficulty level, workshop operations). You may design a special form that participants can fill in anonymously - here is a post-workshop survey template for workshops that pilot a lesson in The Carpentries Incubator. You can use it as a starting point for a post-workshop survey to be shared with your learners at the end of the pilot workshop and you can supplement it with additional questions.
Exercise: homework
The final part of this training will focus on the skills needed to collaborate effectively. Before that there will be a break, during which we would like you to complete the following three tasks:
- Teach one episode of your lesson (probably the one you have been working on in these two days). See the Lesson Trial Runs page for full details.
- After your trial run has concluded (immediately after, or when you
have reviewed any feedback you collected from learners), note down your
answers to the following questions:
- What worked?
- What did not?
- What will you do differently next time?
- What will you change in your material you taught? We will refer to these notes when we reconvene for the last episode of this training.
- Based on your experience teaching the material and the feedback you
received from your learners and helpers, make a list of issues you have
identified with the material you prepared, e.g.:
- examples that did not work as expected,
- improvements that could be made to exercises,
- parts that learners found particularly challenging,
- unexpected questions or misconceptions that came up during the trial run.
What questions do you have? (15 minutes)
The homework from this workshop includes a trial run of one of the episodes you have been developing in your lesson, to a real audience.
After reading the information provided about the trial run task and thinking about this task, what questions do you have about how you should approach teaching that trial run? Is there anything you are unsure of? What resources might help you prepare for that experience?
Key Points
- Spending time on preparing your teaching and feedback collection will make you and your participants get the most out of your workshop pilot.
- Creating clear setup instructions as part of your lesson and circulating them ahead of the pilot is time well-invested and will give you more time when teaching the lesson.
- Instructor Notes are teaching tips that you should include with your lesson to help you (a few months down the line) and other instructors, who have the relevant topic knowledge but have not been involved in the lesson design and development, deliver your lesson more successfully.
Content from Wrap-up
Last updated on 2024-02-22 | Edit this page
Overview
Questions
- What have we learned so far?
- What needs to be done before the next part of the training?
Objectives
After completing this episode, participants should be able to…
- Summarise the lesson development process introduced in this training.
- Describe the trial run task to be completed before the next part of the training
- Identify remaining tasks that should be completed before trialling lesson content.
Recent sections of this training have focused on how to design effective assessments and write a lesson that will be engaging and motivating for learners. We provided further opportunities to apply these concepts to the development of a lesson website with The Carpentries Workbench. Next, we explored The Carpentries lesson development ecosystem and how a new lesson project can fit into that. We finished up with a discussion of the importance of teaching a new lesson and how best to prepare for that experience.
The final sessions will focus on collaboration skills, exploring how trainees can curate their repository and manage their project to foster collaboration and build community around a lesson.
Before that, there will be an extended break during which trainees should conduct a trial run of the lesson content they have been developing in the workshop so far. When the training recommences, there will be an opportunity for trainees to share their experiences and reflections from those trial runs, and to discuss what impact that will have on the next stages of their lesson development.
Polish a Lesson (20 minutes)
This exercise provides you with a chance to look back over everything you have sketched out for your episode(s) and the lesson as a whole and consider what still needs to be done before it can be taught.
You can use this time however you judge will be most beneficial to your preparations for teaching your episode in a trial run.
If you are not sure how to start, consider mapping out the relationships between the objectives of your episode and the examples and exercises via which they will be taught and assessed. For example,
The read CSV and inspect demo supports Objective 2 (load a simple CSV data set with Pandas) and will be delivered using participatory live coding. The objective will be assessed with an exercise that requires learners to apply the read_csv function to another file and count the rows in the resulting DataFrame object.
Does any of your planned content not support any learning objectives?
Is there at least one piece of content planned for each learning objective?
Is there a formative assessment planned for each learning objective?
What do you still need to add/work on?
What can you remove/consider removing?
How will the narrative and example data you have chosen for your lesson support teaching and assessment?
What diagram or other visual aids could you add to supplement your text?
Please spend some time reflecting on what you learned so far and then provide another round of feedback to your trainers.
Organise Your Knowledge (10-15 min)
Take some time to think back on what has been covered so far, then make some notes on the most important points and actions you want to take away from that. The Trainers and other participants will not look at this - it is only for you.
If you do not know where to start, consider the following list for a starting point:
- draw a concept map, connecting the material
- draw pictures or a comic depicting one or more of the concepts
- write an outline of the topics we covered
- write a paragraph or “journal” entry about your experience of the training today
- write down one thing that struck you the most
Please stay in touch as you plan for your trial run.
Key Points
- We have learned about the importance of exercises and data in a lesson, the ecosystem The Carpentries provides for lesson development, and how to prepare to teach a lesson for the first time.
- Before the next part of the training, you should teach a part of your new lesson to a real audience and reflect on the experience.
Content from Reflecting on Trial Runs
Last updated on 2023-09-29 | Edit this page
Overview
Questions
- What did I learn when I taught my lesson?
Objectives
After completing this episode, participants should be able to…
- Reflect on the experience of teaching part of a lesson for the first time.
- Identify changes and improvements that can be made as a result of trialling lesson material.
Welcome back!
In part 2 of the training, we will learn more about the skills and tools you can use to become an effective collaborator on an open source lesson. Before moving on to cover that, we will spend some time reflecting on and discussing the experience of teaching new lesson material for the first time.
In the break since last time, you should have trialled part of the lesson you set out to develop in this training. Congratulations on organising and delivering your first pilot! We will use this session to reflect on that experience, and take some steps to ensure that the feedback and notes you gathered during the trial run are used to improve the quality of your lesson.
Discussion: trial runs (35 minutes)
Look back at the notes you took and feedback you received during your trial run (5 mins).
Now report out to the other participants about that trial run. Try to answer some or all of the following questions:
- What worked well both in terms of content and delivery?
- What did not work as well?
- How close were your time estimates to the actual time needed to teach the material and finish exercises?
- How did the audience perceive the difficulty level of the material?
- What will you do differently next time?
- What will you change in the material you taught?
- What will you change in the way you collect feedback in future pilots?
Moving forward from your first pilot, the next steps to consider are:
- Record all the identified changes and improvements that can be made
to the lesson material and assign a person from the team responsible for
each task. The above exercise provided some reflection points after a
trial run; in terms of lesson content, they may translate into the
following considerations:
- is the material too dense, does it need extra explanations?
- would adding a diagram help explain or curate things better?
- is there too much content for one episode, and do you need to split it into smaller teaching units?
- do you need to re-organise and move some content around to improve the flow and narrative?
- are there enough exercises and practical work?
- do you need to realign your lesson objectives and key messages?
- Decide if for some of the feedback you will not take action. Remember, you do not need to respond to every piece of feedback you receive. For example, it is easy to fall down the rabbit hole of adding extra material just because someone suggested it may be a useful thing - sometimes, adding a link or a callout to extra reading is sufficient. You need to be able to draw a line under any extra modification suggestions to keep you in scope and on schedule.
- Schedule follow-up co-working sessions with your team to carry on working on fixing issues and adding new content to maintain the momentum.
- Add/Update your lessons’ Instructor Notes based on what you learned and to help other instructors who will teach your lesson in the future.
- Think about the timeline for your next pilot(s), even provisionally, to help you set milestones and targets to work towards.
- Consider writing up your pilot experience in a mini blog post that you can share with the community.
Designing and developing quality lessons is hard - there are many pieces of a puzzle that have to come into place: both pedagogical and organisational. Do not be disheartened with the amount and type of feedback you may have received. Even Carpentries lessons that have been around for 10+ years receive improvement suggestions and fixes almost daily. From our experience, bigger lessons that are delivered over a few days require several full pilots before they can even be considered for a beta release. Planning smaller lesson trials (where you test only a portion of a lesson) and doing them more often with a friendly audience from your local groups and close colleagues is more manageable and will help you make steady progress.
Key Points
- “Perfect is the enemy of good” - your lesson does not need to be perfect before you pilot or release it for community review. Early feedback from the target audience will help you avoid straying off your lesson plan.
- Identify changes and improvements you want to make as a result of trialling your lesson and schedule co-working sessions to work on these tasks.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from Collaborating with Your Team
Last updated on 2024-03-25 | Edit this page
Overview
Questions
- How can I use GitHub to manage (report, assign, track, prioritise) tasks of a lesson development project?
- What communication channels are effective for a lesson development team?
- How can I use GitHub to manage contributions from collaborators?
Objectives
After completing this episode, participants should be able to…
- Report and track issues with their lesson using the GitHub interface.
- Review and provide feedback on contributions from collaborators using the GitHub interface.
In this episode we expand on how to use GitHub effectively among a group of known collaborators, currently your lesson development group, on an open source lesson development project, setting the stage for introducing tools and practices for working with newcomers. Here, we explore GitHub features to help you keep track of what needs doing on the lesson, manage contributions and prioritise work. Making decisions and managing your project as a whole will be covered in a follow-up episode on project management. Collaboration skills covered in this and follow-up episodes are transferable to any similar open source project you may be involved with.
Managing Issues
Issues are GitHub’s framework for managing issue/bug reports, feature requests, and lists of future work. They provide a single shared record of all the problems people have found with the lesson, and improvements that could be made, along with solutions and discussions around them. This helps the team to keep track of what they need to work on later, and reduces the chance of receiving redundant reports of issues you already know about.
When you create an issue, you can add a range of details to it. An issue can be assigned to a specific team member - this can be a helpful way to know who is currently working to fix an issue or a way to assign responsibility. You can assign one or more labels to issues to classify or group them. Labels can also be given colours, allowing you to see at a glance from the Issues page the state of your project.
Lesson repositories created from the template are configured with The Carpentries standard set of labels, which are designed to help lesson developers and maintainers to indicate the type and current status of issues. Some of the most frequently used labels include:
-
type:bug- indicates something that is broken in the lesson, e.g. a factual error or broken link -
type:enhancement- indicates that the issue is proposing a way for the lesson to be improved -
type:accessibility- indicates that the issue relates to the accessibility of the lesson -
status:wait- indicates that the issue cannot or will not be worked on until something else is resolved, e.g. the solution to the issue is dependent on another change yet to be made (see also:status:blocked) -
status:waiting for response- indicates that further progress cannot be made until someone (usually the person who submitted the issue) has responded -
status:in progress- indicates that someone is working on a resolution for the issue, preventing potentially redundant efforts from being made by others -
good first issue- indicates that the issue requires relatively little knowledge of the rpoject to fix, making it a good choice for first-time contributors -
help wanted- indicates that a maintainer wants help on an issue or pull request
GitHub provides a default set of labels for repositories which can be used “as is” or customised (as done by The Carpentries above) - check the GitHub documentation on Issues for the full reference. You can also create your own custom labels to help with classifying issues.
If you are developing your lesson in The Carpentries Incubator,
you can register the repository for inclusion in The Carpentries Help Wanted Issues
page, which will list all issues labeled with
help wanted and/or good first issue. Issues
listed on this page are more visible to the wider Carpentries community,
making it a good way to attract spontaneous contributions from
newcomers.
Exercise: creating issues (15 minutes)
After your trial run, you should have created a list of issues you identified with your lesson. Open issues on the GitHub repository for your lesson, for each item on that list. Remember to add labels to each issue according to its type, priority, etc. As you create them, you may also want to assign these issues to a member of your lesson development team.
Managing Contributions
Although it is possible (for collaborators with sufficient access to
the project) to make changes to a lesson directly on the
main branch of the repository, doing so is not
recommended:
- It increases the likelihood of unintentional conflicts being created when two collaborators work on the same file simultaneously.
- It does not provide an opportunity for changes to be checked and discussed before they are included in the lesson. This makes it much more likely that mistakes will be included in the lesson, and that the commit history of the project will include many changes made only to fix/polish/undo previous commits.
- It prevents your team from benefiting from the powerful features that Git and GitHub provide to support and foster collaboration.
Instead, it is recommended practice to do most work in
branches: sets of changes that build on the work in the
main branch (or another) but are not included in that
branch until they are explicitly merged into it. GitHub
provides a way for this merging process to be managed, by giving a
platform for the changes to be described, discussed and reviewed, before
a decision is made to integrate or not integrate the changes in the
default branch of the project and the associated lesson. On GitHub,
these units of discussion, review, and acceptance/rejection of the
changes within a branch are called Pull Requests.
Why “Pull Request”?
The term Pull Request originates from the early days of GitHub, when a contributor would be requesting that the project maintainer(s) use Git to fetch (download) their branch and merge the changes into the main branch of the project. In the terminology of Git, this combination of fetching and merging a set of changes is called pulling. The name Pull Request is slightly anachronistic now, because a project maintainer can often manage the whole process of reviewing a contribution in GitHub’s web interface.
Note: A common nickname for a Pull Request is PR or PRs for Pull Requests.
Submitting and Reviewing Pull Requests on GitHub
The Trainers will collaborate on a demonstration of the process laid out below.
Pull Request Activity
Note: This activity is designed to be carried out by two Trainers acting in two different roles: “(S)” denotes the Pull Request Submitter role, “(R)” the Pull Request Reviewer.
- (S) Explore list of issues on repository, choose issue to fix, make note of issue number.
- (S) Edit a file to fix the issue, commit changes to a new branch with an appropriate name. Ensure that the changes include some typo etc that the Reviewer will be able to comment on and/or suggest a fix/improvement to.
- (S) Show how to open a pull request from that new branch to
main. - (S) In Pull Request description, reference the number of the issue being fixed, using a keyword “Fixes #X” or “Resolves #X” (where X is the issue number) to ensure automatic closure.
- (S) Demonstrate how to mark the PR as a draft, and then how to mark as it as ready for review.
- (S) Request a review from the Pull Request Reviewer.
- (R) Open the new Pull Request in GitHub.
- (R) Explore the diff of the PR, by opening the Files changed tab and talking through the display.
- (R) Comment on a line in the diff.
- (R) Suggest a change to the diff, using the “Add suggestion” button
or three backticks marked
suggestion. - (R) Complete your review by adding some summary text and selecting “Request changes”.
- (R) Return to Conversation tab, to show how the PR thread looks after the review has been submitted.
- (S) Navigate to PR and the Files changed tab.
- (S) Respond to Reviewer’s comment(s), accept their suggested changes (draw attention to the option to batch multiple suggestions into a single commit).
- (S) Return to the Conversation tab and request a new review from the Reviewer.
- (R) Look again at the PR diff (Files changed) and ensure that requested changes have been made.
- (R) Approve the PR.
- (R) Thank the Submitter for their contribution.
- (R) Merge the PR.
- (R) Navigate to issue listing to confirm that the relevant issue has been automatically closed.
Practice with Pull Requests (15 minutes)
In breakout groups, assign yourselves the role of Submitter (S) and Reviewer (R), and repeat the process described above by completing the steps below. If there are any issues open on the repository that can be solved very quickly, feel free to address those with this exercise. Otherwise, the Submitter can open a Pull Request to make simple changes: typo fixes, or some completely arbitrary changes to a file. If the changes made for the exercise are not improvements to the lesson, the Reviewer should follow all the steps but close the PR at the end instead of merging it.
- (S) Edit a file and commit the changes to a new branch.
- (S) Open a new Pull Request from this new branch to
main. If your PR will fix an open issue on your repository, include “Fixes #X” at the start of the PR description, where X is the issue number. - (S) Request a review from the Reviewer.
- (R) Review the PR, making comments and suggesting changes as appropriate. When you have finished, approve the PR or request changes, based on your review.
- (S) Respond to any comments from the Reviewer and/or, if any changes were requested, make those and re-request a review when you have finished.
- (R) When all changes have been made to your satisfaction, approve the PR, thank the Submitter for their contribution, and merge the PR (Or close it if the changes were made for practice but should not be included in the actual lesson.)
ProtectingmainBranch
Repository administrators can enforce this workflow by
protecting the default branch of the GitHub repository:
requiring anyone without Admin access to submit changes by pull
request instead of committing changes directly to main.
The GitHub documentation provides more information about this feature and other branch protection rules.
Reviewing is a Skill
Making effective review comments in a PR is something you will get better at with practice - here is some general guidance to help with reviewing a PR from a collaborator:
- Good reviews have empathy - consider the contributor’s feelings, identify positives and try not to be judgemental.
- Aim for improved/better rather than perfect.
- Make sure your review comments are specific and actionable.
- Make it clear in the comment what you want changed as part of a PR and, ideally, provide a concrete suggestion.
- Only request changes if the content is wrong or hard to understand (remember, different does not always mean better).
- If a conversation on a PR hasn’t been resolved by one or two back-and-forth exchanges, consider scheduling a meeting to discuss further.
The Turing Way has more on reviewing contributions, and a lot of the advice in Alex Hill’s blog post, The Art of Giving and Receiving Code Reviews (Gracefully) also applies to reviewing prose.
If you need more help or practice with GitHub features, you can attend a GitHub skill-up session for Carpentries Maintainers and lesson developers which uses the following curriculum. These sessions will be periodically announced on The Carpentries calendar of events, The Carpentries Slack and mailing lists.(https://carpentries.github.io/github-skill-up-maintainers/).
Managing Communication
Having an open, publicly-visible list of all the issues with your project is a helpful way of letting people know you are aware of issues and you are working on them. This can indicate to an external audience that the project is active. It also provides you and your collaborators with an “at a glance” view of the state of the project, making it easier to prioritise future work.
GitHub provides a useful notification feature for collaborative work
- mentions. The mention system notifies team members when
somebody else references them in an issue, comment or pull request - you
can use this to notify people when you want to check a detail with them,
or let them know something has been fixed or changed (much easier than
writing out all the same information again in an email!). You can use
the mention system to notify individual GitHub accounts or whole teams
(when you need to mention multiple people at once and it is not
practical to list them all individually). Typing @ in
GitHub will bring up a list of all accounts and teams linked to the
repository that can be “mentioned”. You can also mention people who
aren’t linked with the repository but they won’t be listed in the
@ drop-down list so you will need to know their GitHub
username in order to mention them.
Slack is commonly used in the Carpentries community for quick, day-to-day message exchange among teams. You can create a channel for your lesson development project under the Carpentries public Slack workspace or if you need more complex team communications you can create your own Slack workspace for free. Note that the Carpentries Slack is an enterprise workspace so all messages and files will be retained and no messages will be lost (for free workspaces only the most recent 10,000 messages can be viewed and searched and file storage limit is 5 GB).
Collaborators can also use other platforms to discuss lesson development or receive contributions from newcomers who are not yet fluent in using GitHub’s systems of communication. The Carpentries can assist with creating a mailing list specific to the development of your lesson on their TopicBox platform for managing threaded email discussions. Also make sure to join the Incubator lesson developers mailing list on TopicBox to keep an eye on announcements and discussions relating to lesson development in general within the Carpentries community.
Recommended Reading
Here are some additional resources where you can learn more about collaborating with your team in an Open Source project:
- The Guide for Collaboration section of The Turing Way
- The Communication resources section of the Contributor Experience Handbook, especially the collection of resources on productive conflict
Key Points
- GitHub’s features Issues, Pull Requests and Mentions can all be used to transparently and effectively communicate and collaborate in a team working on a lesson development project.
- Slack and emailing lists provide communication channels outside of GitHub - for quick day-to-day messaging and more permanent discussions around lesson issues and development, respectively.
Content from Break
Last updated on 2023-08-21 | Edit this page
Take a break. If you can, move around and look at something away from your screen to give your eyes a rest.
Content from Collaborating with Newcomers
Last updated on 2024-03-25 | Edit this page
Overview
Questions
- What makes a project welcoming for newcomers?
- What information should I provide about the project to potential collaborators?
- How can I ensure regular progress on lesson development?
Objectives
After completing this episode, participants should be able to…
- Adjust a lesson repository to attract potential new collaborators.
- Create and modify issue and pull request templates in GitHub.
- Identify strategies to encourage regular development and maintenance of a lesson.
You may start working on projects by yourself or with a small group of collaborators you already know, but open source projects in the Carpentries community often grow beyond that small group. Community collaboration is the cornerstone of The Carpentries - all curricula and various programmes are developed together by the community members across borders, domains and backgrounds. Most initiatives are done through community consultation and collaboration is used as a pathway to empower people, give members a sense of purpose in the community and realise shared goals.
Collaboration helps bring different perspectives to solving a problem and share the load of the required work to develop, pilot, and, later on, maintain the lesson. People drop in and out of projects for various reasons and their levels of involvement may vary over time due to other commitments - for this reason it is worth taking time from the outset to make a project welcoming for newcomers in order to achieve a steady number of people working on your project at any time. You should plan to make it easy for new collaborators to join, to set up a local workspace so that they can contribute, help them find tasks so that they know what to contribute, and make the contribution process clear so that they know how to contribute. You also want to make it easy to give people credit for their work on your project as well as for others to reuse and give credit to your project. In this episode, we cover best practices for collaborating with newcomers and tools available to help. And remember, anything you do to make it easier for newcomers to contribute will be universally useful for all collaborators.
Documenting Your Lesson
For collaborative projects, it is important to spend the time on ‘external-facing’ features, such as documentation, to make the project as welcoming and as easy to get involved in as possible for newcomers who may not have a deep grasp of the project or know all its existing problems. Such documentation will be useful to yourself and other team members as well, e.g. if you are trying to come back to the project after a break or are reusing it for a new collaboration in the future.
Your lesson documentation should contain the following information, which should be kept up to date.
Description of the Project - README File
README, README.txt or
README.md is a plain text or Markdown file located in the
project’s root directory representing the default landing or home page
for repositories in GitHub (and similar project repositories) by
convention. It represents the first piece of documentation that people
will see when visiting your lesson repository, hence it should concisely
explain what the lesson is about and who it is for, and contain links
and pointers to further information. For example, README
should include:
- lesson title
- lesson description
-
rendered version of the lesson - even though this
is typically included in the
Aboutsection of the repository, it is always useful to link to the URL where the rendered lesson is available - contact information - include up-to-date email addresses or mailing lists or other details on how to get in touch with the lesson maintainers
-
contributing information - this is an opportunity
to list what kinds of contributions are sought (and what are not) and
how to get involved in lesson development for new contributors. You can
provide more details in a separate
CONTRIBUTINGfile within the repository’s root directory and link to it fromREADME -
credits/acknowledgements - make sure to credit
those who have helped in the lesson’s development or inspired it, and/or
any resources/templates that you have reused. You can also link to a
separate
AUTHORS.mdfile within the repository’s root directory to list all the people who contributed to the lesson content (if this list starts to become too large to include in theREADMEitself) -
citation - a convention is to include the citation
information for your lesson in a separate
CITATIONfile within the repository’s root directory (and link to it fromREADME) so others can cite the use of the lesson in their own publications and media and it can also be automatically discovered by other applications. Citation can be a Digital Object Identifier (DOI) issued by a reputable DOI-issuing repository such as Zenodo, which should be obtained as a permanent identifier as soon as your lesson reaches the beta stage, or an appropriate academic publication once your lesson becomes stable and is peer-reviewed. Citation can be contained in a plain text file (CITATION,CITATION.txt), a Markdown file (CITATION.md), or in the recently adopted Citation File Format (CITATION.cff). The CFF is defined using YAML (which we already encountered in theconfig.yamlfile for the lesson website) and is now the recommended way of storing citations in GitHub. Advantages of using CFF are that GitHub will automatically show the citation information in the sidebar, making it more visible and accessible for visitors to your repository. In addition, metadata fromCITATION.cfffiles will automatically be used by Zenodo when registering the DOI for your lesson/project. CFF metadata is also recognised by the Zotero reference manager. -
license - a short description of and a link to the
lesson’s license typically contained in a separate
LICENSE,LICENSE.txtorLICENSE.mdfile within the repository’s root directory. The lesson repository created from the Carpentries lesson template already contains a defaultLICENSE.mdfile, but you should modify this to more accurately describe how the lesson content can be re-used by others.
Instructions for Contributors - CONTRIBUTING File
CONTRIBUTING, CONTRIBUTING.txt or
CONTRIBUTING.md is a text or Markdown file within the
repository’s root directory where you should provide detailed
description of the ways people can send their contributions, what kinds
of contribution will be credited and in what ways. For example, at what
point would someone be listed as an author and how you will credit
contributions that are not recorded in the commit history of the
project. The latter is particularly important for newcomers who may not
be proficient with the use of GitHub’s features for collaborative work,
such as issues, mentions, pull requests and code review, but could still
provide valuable input and contributions.
Carpentries lesson repositories already have a generic Carpentries
CONTRIBUTING.md file to get you started. You should review
it and, if need be, modify it to fit your team’s needs and way of
working. For example, you may want to expand on what contributions you
are not looking for if your lesson already contains more
material than can be covered in a typical workshop. You can also detail
all channels on which you are willing to receive contributions on, if
they are different from the defaults included in the file.
Issue and Pull Request Templates
Consider setting up issue and pull request templates to help newcomers who may not have much experience working on collaborative projects in GitHub. Such templates can provide a structure for the issue/pull request description, and/or prompt them to fill in answers to pre-set questions. Both can help contributors raise issues or submit pull requests in a way that is clear, helpful and provides enough information for maintainers to act upon (without going back and forth to extract it). GitHub provides a range of default templates, but you can also write your own.
Other Documentation
Once you have set up the basic documentation about your lesson, you may consider adding the following useful information to your documentation too:
- set-up guides for installing and rendering the lesson locally. In most cases, a link to The Carpentries Workbench documentation will be sufficient. You can also contribute to the Workbench documentation to help improve it for the community.
- how to create and modify the pages in the lesson
Exercise: preparing your repository for collaboration (15 minutes)
Spend some time doing one of the following:
- Modify the
README.mdandCONTRIBUTING.mdfiles in your repository to provide the relevant information for would-be contributors to your lesson:- the name(s) of the current developer(s), linked to their GitHub profile(s) (README)
- contact information (README, CONTRIBUTING)
- guidance on how people should contribute, and what kinds of contribution you are most interested in receiving (CONTRIBUTING)
- links to any important background information (README)
- links to any funding bodies or host institutes that are supporting the development of the lesson (README)
- Create a new issue or pull request template, or modify an existing one, to guide contributors on how best to begin collaborating with you on GitHub.
Groups of collaborators taking this training together should discuss first how they will assign these tasks between them.
Collaborating on a Lesson
Boosting the Visibility and Attracting New Collaborators
In addition to having the complete documentation in the lesson repository, The Carpentries community provides a number of ways to further raise the visibility of the lesson among the broader community and encourage community members to contribute to its further development. For example:
- listing issues from the lesson repository on The Carpentries Help Wanted page
- featuring your lesson in the Incubator Lesson Spotlight
- writing a blog post about the lesson for The Carpentries Blog, and/or attending a community discussion call to promote the lesson
- advertising the lesson at various Carpentries mailing lists - e.g. general discussion, instructors, regional communities or specific curriculum lists
You should also consider including your lesson in Hacktoberfest and similar wider community open source events aimed at encouraging people to contribute to open source projects periodically throughout the year.
Issue Labelling for Newcomers
You can encourage contributions to your lesson from newcomers by
using specific labels on issues to highlight suitable opportunities to
help. Apply the good first issue or
help wanted labels to issues in your repository to indicate
that the maintainers will particularly welcome help or pull requests
fixing such issues.
Noticing When Something Happens
In addition to Mentions, GitHub implements a comprehensive notifications system to keep you up-to-date with activities in the lesson repository so you can react in a timely manner. Check out GitHub’s documentation on setting notifications on individual repository. You can choose whether to watch or unwatch all events on an individual repository, or can choose to only be notified of certain event types such as issues, pull requests, mentions, etc.
Managing GitHub Notifications
If you work on multiple projects, or the projects you follow on GitHub are particularly active, the volume of notifications you receive can quickly become overwhelming. Here are some approaches you can take to help you stay on top of things, and distinguish the high-priority tasks and important updates from the regular traffic.
Email Notifications
If you want to filter, organise, and redirect email notifications from GitHub, here are some characteristics of the messages that you make use of:
- All GitHub notifications are sent from the address
notifications@github.com. - Notification emails are sent to the address
repo-name@noreply.github.com, whererepo-nameis the name of the repository where the notification was triggered. - The email subject begins with
[org-or-user/repo-name], whereorg-or-useris the name of the organisation or the username of the user who owns the repository andrepo-nameis the name of the repository where the notification was triggered. - In addition to your email address, the cc field of the message
contains an address that describes the type of event that triggered the
notification, e.g.
author@noreply.github.comfor activity on an issue or pull request that you opened,mention@noreply.github.comfor a mention of your username, orteam-mention@noreply.github.comfor a mention of a team you are a member of, etc. - The email header (metadata) includes a
mailing-listfield with an identifier in the formrepo-name.org-or-user.github.com, which can be used to filter by the project and/or its owner.
Most email clients provide configuration for rules that can be set to redirect messages to particular folders, and/or to annotate them with a mark or flag, based on this kind of information. Here is the documentation for setting such rules in Gmail.
Notifications on GitHub.com
The alternative to using email to keep track of project activity is to manage notifications on GitHub. Adjust how you receive notifications in the Notifications section of your account settings. When logged in, you can visit https://github.com/notifications to see notifications for your account, presented as a table. Through this interface, you can:
- group notifications by repository.
- see notifications for a particular repository.
- see particular types of notification e.g. mentions, issue assignments, etc.
- create more sophisticated filters using the search bar, e.g. all mentions for all repositories in a particular organisation.
The account notification settings allow you to specify certain types of notification that should be sent by email, while everything else is collected in the web interface. Whatever strategy you choose, one of the most important habits that will help you stay on top of your projects and tasks is to remember to check these notifications frequently.
Saying “No”
Not every contribution is a good fit for a project and, especially as your lesson becomes more stable, you probably will not want to accept every suggested change. If you receive a pull request or issue that does not fit to your lesson, consider the following points to help you politely decline without demotivating the contributor from contributing to your lesson (or another open source project) again:
- Thank them for taking the time to contribute.
- Explain why the contribution does not fit into the lesson, and offer suggestions for improvement if you’re able to.
-
Link to relevant documentation, if you have it.
Notes about the design of the lesson (e.g. from this training) and any
relevant discussion threads can be very helpful here. If you notice
similar repeated requests/contributions, you might want to address them
in your documentation (e.g. in the
CONTRIBUTING.mdfile) to save yourself time in future. - Close the request.
The advice above is taken from The Carpentries Maintainer Onboarding curriculum. That resource, and the Best Practices for Maintainers guide from GitHub on which it is based, make excellent further reading as you prepare to transition from your role as an active developer into another as a responsive and responsible maintainer of your lesson.
Making Progress
The following practices have been shown to help maintain steady progress with lesson development:
- being responsive to notifications about activities and mentions
- scheduling regular co-working/sprinting sessions with team members (e.g attaching your sprint sessions to other open source community activities, which may offer goodies, rewards and prizes for participants, can provide motivation and activity spikes)
- working alongside other members of The Carpentries community at Maintainer or lesson development co-working sessions
- blocking time in your calendar for issue triage/solo material writing
- planning lesson pilots in advance to help set targets
Make sure to join the Incubator lesson developers mailing list to keep an eye on announcements and discussions relating to lesson development in the Carpentry community.
Exercise: planning to collaborate (10 minutes)
Use this time to plan how you will continue to make progress with developing your lesson, and ensure that you are ready to respond to external contributors when they arrive at your lesson. This time is yours to do with as you think is best for yourself and your project, but you might consider using it to do one or more of the following:
- schedule a lesson development co-working session with your collaborators, and/or mark the next community co-working session in your calendar.
- ensure that your the notification settings are appropriately configured for your lesson repository.
- contact The Carpentries team to ensure issues from your lesson will be included on the Help Wanted page.
Recommended Reading
Here are some additional resources where you can learn more about encouraging newcomers to get involved with your Open Source project:
- Ten simple rules for helping newcomers become contributors to open projects by Dan Sholler, Igor Steinmacher, Denae Ford, Mara Averick, Mike Hoye, and Greg Wilson
- The Contributor Experience Handbook, especially the following sections:
Key Points
- “If a project doesn’t make a good first impression, newcomers may wait a long time before giving it a second chance” - Karl Fogel, the author of “Producing Open Source Software: How to Run a Successful Free Software Project”.
- You should make an active effort to attract potential collaborators and try to make them all feel welcome and included. The Carpentries Help Wanted page and featuring in the Incubator Lesson Spotlight can boost the visibility of your lesson. Creating the appropriate documentation and using GitHub features such as labels, and issue/pull request templates will help lower the barriers for contributions to your project.
- When you design your lesson with new contributors and increased accessibility in mind, you make things better for everyone in the process.
- Scheduling regular co-working sessions, blocking time in the calendar for issue triage, and setting and being responsive to GitHub notifications will ensure regular progress on the lesson.
Content from Project Management and Governance
Last updated on 2024-03-25 | Edit this page
Overview
Questions
- How can I use GitHub to manage a lesson development project?
- What strategies exist to help collaborators make decisions and govern an open source project?
Objectives
After completing this episode, participants should be able to…
- Use GitHub features to help you with planning and managing your project.
- Prepare for a decision-making meeting between collaborators.
Project planning & management
Developing a lesson is a project and, like most projects, it consists of multiple tasks. Keeping track of the list of tasks the team has to do, progress on each, prioritising tasks for future development, sprints and releases, etc., quickly becomes a non-trivial task in itself. Without a project management tool/framework, it can be hard to keep track of what’s done, or what needs doing, and particularly difficult to convey that to others in the team or share the responsibilities.
Different tools and platforms exist to help you with project planning and management - for example, GitHub’s Milestones and Projects, Asana, Trello, Miro, even sticky notes on a white board. Different tools will work best for different teams of collaborators, but making sure you use any tool to plan and manage work on your lesson development project is better than making it up as you go along.
Project Management in GitHub
Carpentries lessons are developed in GitHub so it is convenient (but not a must) to use GitHub’s Milestones and Projects to help you keep track of different components of the project and their current status.
Milestones are a way of organising issues on your project into smaller units of work (e.g. deliverables) that have deadlines and progress of which needs to be closely tracked (e.g. release management). Projects provide a way of visualising and organising the work on a higher level and are more suitable for project management tasks that are more continuous and not time-bound. While a Milestone is associated with a single repository, a Project can span over multiple Milestones and even different repositories within an organisation in GitHub.
Milestones vs Projects
For a good description of differences between the two, have a look at this answer on StackOverflow.
A Milestone answers the question:
“What is remaining to finish off this task?”
A Project answers the question:
“What (tasks) are we working on at the moment?”
Milestones are good for managing time-bound, structured iterations and working in short sprints on batches of issues. Dedicated sprints seem to work well for lesson development, so we will have a look at how Milestones can help with this. We recommend you to also have a look at GitHub’s documentation on Projects and see if they are suitable for your lesson development workflow.
GitHub’s Milestones
A Milestone is a collection of issues to be worked on in a given time period. Milestones are good for tracking progress as they provide an ordered list of issues which must all be resolved for the overall task to be considered complete. Milestones work like labels - they mark and group issues that are expected to be closed at some point in time.
The Milestones page in GitHub is accessible from the
Issues or Pull requests pages - from there you
can create a new Milestone, set its description and due date, sort all
Milestones by due date and see the percentage of issues completed for
your Milestones. Within an individual Milestone’s page, you can see the
list of associated issues and prioritise them by pointing to the left of
an issue until the ‘hand’ pointer appears and then drag-and-dropping it
to a new location in the list (with higher position in the list
indicating higher priority). Back on the
Issues/Pull requests page, you can assign
issues/pull requests to a Milestone and filter your issues/pull requests
by Milestones.
Exercise: prioritising issues & planning work (15 minutes)
Use Milestones to prioritise issues and plan the first sprint and/or the first lesson release with your team.
Depending on how many issues were registered on your repository, it is possible that you will not be able to work on all the issues in the first sprint. As the development of your lesson progresses, any remaining unresolved issues can be reconsidered and prioritised for another sprint/milestone.
This process of receiving new issues and requirements, prioritisation, and working on them is naturally continuous - with the benefit that at key stages your team is repeatedly re-evaluating what is important and needs to be worked on which helps to ensure real concrete progress against project goals and requirements which may change over time.
Project Governance
In addition to managing your project on a day-to-day basis, you should consider a governance model for your lesson to describe the ground rules of participation, the roles that project participants can take on and the process for decision making within the project. You may wonder why you’d ever need a formal governance process - after all, everyone in your team is collegial, polite and hard-working. However, even among the most harmonious teams you will likely realise you need an agreed way to make decisions and resolve conflicts when your team encounters their first real disagreement1.
The moment your collaborator group exceeds 3 or 4 members (and fewer if the collaborators do not all know each other equally well at the outset) you should establish and document some kind of governance process. It will help mitigate any power imbalances which are expressed when one team member (or a group of team members) are more vocal and are able to dominate the decision-making process. It is advised to make these considerations early on, before your project grows too much and introducing structure and process into it becomes more difficult and complicated. Besides, if you do not establish these processes from the start, you are likely to discover the need to do so only when you first encounter disagreement within the team: a situation unlikely to provide perfect conditions for a discussion of governance.
Here are some aspects of governing a project that you should consider. These are borrowed from the Working in teams chapter of Research Software Engineering with Python, a book on how to work productively in small teams where everyone is welcome:
- Codes of Conduct
- meeting rules
- decision-making process
- handling conflict
- making all the above obvious to newcomers
For some other examples and inspirations of governance processes, consider the following:
- Martha’s Rules for discussing and deciding on issues in group meetings (if you cannot access the publication, see Greg Wilson’s explanation of Martha’s rules)
-
GitHub’s Minimum Viable
Governance (MVG) for establishing a lightweight governance process
into free and open source projects that are run in version control
systems.
- In MVG decisions are made through consensus of the project maintainers or determined based on the maintainers’ consideration of a number of factors in “good faith” (when explicit agreement of all maintainers cannot be reached). In reality, consensus-based governance can run you into trouble when you hit an issue where maintainers do not all agree but still want to make progress. Also, the definition of “good faith” is open to interpretation and leaves plenty of room for people, especially the powerful, to get away with acting in bad faith.
- A system based on voting and majorities may help you get around this issue. MVG is designed for a one organisation-many projects setup and is too elaborate for a single lesson project. However, it includes “boilerplate” template files for a steering committee, charter, governance, Code of Conduct documents, and language laying out a preference for consensus but with voting as a fallback, and other templates that may be helpful for those starting out and unsure about how to get on with establishing a governance process.
- This list of resources related to community governance from The Center for Scientific Collaboration and Community Engagement provides a collection of links to further reading material on this topic. It includes various guides, blog posts, and examples of community governance that may inform the discussions and decisions for your project.
Exercises: decision making process (15 minutes)
These activities are intended to help you think about the human relationships between collaborators on a project and how your interactions can influence the progress you make.
As a team read the description of project governance for this training curriculum.
Discuss with your colleagues which, if any, aspects of this would work for your team and which would not. What model of governance and decision-making might work for your team?
If you have time remaining, discuss what factors influence how much power each member of your team has in the context of the project. This could be “hard”/formalised power, e.g. project collaborators in more or less senior positions within the same workplace, and/or “soft”/informal, e.g. differences in the amount of time each collaborator has available for the project, or different levels of expertise in the subject matter of the lesson, or the lesson infrastructure and associated tools such as Git and GitHub.
Write down any questions you may have for your trainers.
(Recommended reading: the Dealing with Disruptive Behaviours resource linked in the exercise for lesson developers working alone, below, may also be helpful to ensure smoother project progress in your team.)
The Dealing with Disruptive Behaviours resource from the US National Oceanic and Atmospheric Administration (NOAA) Office for Coastal Management provides guidance for meeting facilitators. It includes examples of several common behaviours that can be disruptive to meetings (and therefore to efficient progress on a project) and advice for facilitators on how to intervene and mitigate them.
- Open the Dealing with Disruptive Behaviours resource, select each behaviour, and read the descriptions.
- Which of these behaviours do you recognise in yourself? Put another way, which “character” (“Talkative Blowfish”, “Apathetic Flounder”, etc) do you most resemble?
- Think about another project you work on where you are part of a team. Can you identify any of the disruptive behaviours in your colleagues? If so, can you find anything in the suggested interventions for these behaviours that could help you in your shared project?
Recommended Reading
Here are some additional resources to learn more about Open Source project management/governance:
- Producing Open Source Software: How to Run a Successful Free Software Project by Karl Fogel
- Program Management for Open Source Projects by Ben Cotton
- The Shared Ownership in Open Source Projects section of The Turing Way
- A guide to open source project governance models from Red Hat
Key Points
- Being intentional about task/project management for your lesson development project is better than making it up as you go along.
- Different tools and platforms exist to help you with project management - chose one that suits your team best.
- Dedicated work sprints seem to work well for lesson development and GitHub’s Milestones can be used effectively to plan and manage such sprints.
- Make sure to consider and establish a governance model for your project early on - to describe the ground rules of participation and the process for decision making within the project.
The _Challenges with Applying Ownership Models Retroactively section of The Turing Way provides explores further challenges with trying to introduce governance structures to an established Open Source project.↩︎
Content from Final Wrap-up
Last updated on 2024-03-01 | Edit this page
Overview
Questions
- What steps can you take to be a good collaborator?
- How can we improve this training?
Objectives
After completing this episode, participants should be able to…
- Draw a diagram or compose a written summary of the key concepts from this training.
- Articulate constructive feedback.
We have reached the end of this training.
Looking Back
Throughout this training we promoted a collaborative approach to lesson development (as many other initiatives in The Carpentries community are conducted). Collaboration helps bring different skills and perspectives to a project and share the load of the work required to bring the project successfully past the finishing line. For lesson projects, this shared effort also extends to maintaining and updating the lesson after the initial development phase.
Exercise: What steps can you take to be a good collaborator? (20 minutes)
Spend some time reflecting on collaboration aspects we covered in this training and exploring at least two of the following lesson repositories, noting down what things you noticed in these repositories that could aid collaboration:
- Collaborative Lesson Development Training (this lesson)
- Introduction to Working with Data (Regular Expressions) official Library Carpentry lesson
- Introduction to Geospatial Raster and Vector Data with Python lesson in Carpentries Incubator
- Introduction to R for Metagenomics lesson in Carpentries Lab
Trainers will then lead a discussion on what people found and steps you could take to improve your lesson repository.
Collaboration Top Tips
While you will know the best collaboration practices for your team, here are some suggestions:
- Provide ‘external-facing’ features for your lesson (such as description, objectives, contribution and ‘how to get involved’ guides, preferred way of communication) - these will be universally useful for everyone coming to your repository (your team included).
- Find the best tools that work for your team for communication, documentation and knowledge sharing, issue & project management.
- Document and share knowledge within the team to facilitate team collaboration. Share decisions and communicate with your team synchronously and asynchronously.
- Plan your meetings to get the most out of your team (e.g. set agenda, assign meeting roles - chair, note-taker, time-keeper, etc.) and be appreciative of people’s times.
- Record and manage issues with your lesson as you develop, trial and maintain it (e.g. assign people responsible for issues, categorise and prioritise them).
- Assign tasks and roles to distribute the responsibilities - this includes assigning larger roles (project management & governance) to team members and assigning specific issues/tasks to collaborators. Progress is much more likely to be made when people know what they and others should be working on.
- Dedicated sprints where the whole team works on a set of issues towards a milestone work well for lesson development and making progress on projects in general.
- Be open and welcoming to new contributors and provide different opportunities to get involved with your project (from novice to expert collaborators). Point to upskilling guides (or run upskilling sessions if possible) to bring people up to speed.
- Manage contributions in a timely manner by noticing when things happen and responding to new issues, comments and Pull Requests (PRs). Show appreciation for everyone’s contributions.
- People bring different perspectives, expertise and skills to a collaborative project and, while they can be productive and harmoniously complement each other, having some decision making and conflict resolution process in place will make sure progress can be made when disagreements arise.
Looking Ahead
By now, you should have made good progress towards a first draft of your lesson.
Referring back to the life cycle stages for lessons described earlier in the curriculum, your lesson is still in pre-alpha but hopefully approaching alpha, when it is ready to be taught in full for the first time.
As you and your collaborators continue to iterate over the lesson - teaching it, gathering feedback, reflecting on the experience, and improving it for next time - the lesson will pass through alpha into the beta and stable stages (perhaps via peer review in The Carpentries Lab).
Your role as the original developers of the lesson, and the kind of interactions/contributions taking place on your repository, will change as you move through these stages.
| Life cycle stage | Type of contribution | Your role |
|---|---|---|
| pre-alpha | new content and whole new episodes; changes/additions to lesson objectives | active development; onboarding new collaborators; project management |
| alpha | improving existing content; adding/expanding/removing content; polishing changes/typo fixes | active development; reviewing changes; documenting feedback/actions based on pilot workshops; project management |
| beta | feedback from pilot workshops; polishing changes; expanding instructor notes; suggestions for major overhauls/reorganisations | recruiting pilot instructors; reviewing changes; guiding contributors |
| stable | feedback from workshops; polishing changes; new/updated content | reviewing changes; guiding contributors; onboarding new maintainers; stepping away from the project |
Farewell and Feedback
Thank you for following this training. We hope you found it helpful to build a foundation for your current lesson project and to prepare you to collaborate on other projects in the future.
To bring the training to a close, the Trainers will lead a number of activities to gather feedback and help you reflect on the experience.
One Up, One Down (10 min)
Provide one up, one down feedback on the entire Collaborative Lesson Development Training. Remember:
- Say only one thing, and try not to duplicate. This gets harder for those who come later!
- Trainers should try not to respond, only record responses (e.g. in the Etherpad). This is also hard, but important!
Reflection exercise (15 min)
Take this time to reflect on what you have learned in this last part of the training, and take some action to plan to successfully complete the development of your lesson after you leave. It is up to you how you use this time, but you might find some inspiration in the prompts below.
- How could you use the project management features of GitHub to prepare for the next stage of the lesson development process?
- If you have been working in a collaborative group, what steps could your group take to improve the way you work together for the remainder of the project?
- If you have been working on the lesson alone so far, what steps could you take to find new collaborators to help you with the further development of your lesson?
- Based on the experience of teaching a section of your lesson, do the objectives you wrote for your lesson still seem realistic? Can you identify anything that can be added or removed now that you have completed a trial run?