Lesson Title

Lesson Design

Overview

Teaching: min
Exercises: min
Questions
  • How do we design lessons?

Objectives

Key Points


GitHub, Markdown, and Jekyll

Overview

Teaching: 10 min
Exercises: 0 min
Questions
  • How are pages published?

Objectives
  • Explain how GitHub Pages produce web sites from Git repositories.

  • Explain Jekyll’s formatting rules.

This episode describes the tools we use to build and manage lessons. These simplify many tasks, but make other things more complicated.

Repositories on GitHub

Our lessons are stored in Git repositories (or “repos”) on GitHub. We use the term fork to mean “a copy of a GitHub-hosted repo that is also hosted on GitHub” and the term clone to mean “a copy of a GitHub-hosted repo that’s located on someone else’s machine”. In both cases, the duplicate has a reference that points to the original repo.

In an ideal world, we would put all of the common files used by our lessons (such as the CSS style files and the image files with project logos) in a template repo. The master copy of each lesson would be a fork of that repo, and each author’s working copy would be a fork of that master:

Forking Repositories

However, GitHub only allows a user to have one fork of any particular repo. This creates a problem for us because an author may be involved in writing several lessons, each with its own repo. We therefore use GitHub Importer to create new lessons. After the lesson has been created, we manually add the template repository as a remote called template to update the lesson when the template changes.

Repository Links

GitHub Pages

If a repository has a branch called gh-pages (short for “GitHub Pages”), GitHub publishes its content to create a website for the repository. If the repository’s URL is https://github.com/USERNAME/REPOSITORY, the website is https://USERNAME.github.io/REPOSITORY.

GitHub Pages sites can include static HTML pages, which are published as-is, or they can use Jekyll as described below to compile HTML and/or Markdown pages with embedded directives to create the pages for display.

Why Doesn’t My Site Appear?

If the root directory of a repository contains a file called .nojekyll, GitHub will not generate a website for that repository’s gh-pages branch.

We write lessons in Markdown because it’s simple to learn and isn’t tied to any specific language. (The ReStructured Text format popular in the Python world, for example, is a complete unknown to R programmers.) If authors want to write lessons in something else, such as R Markdown, they must generate HTML or Markdown that Jekyll can process and commit that to the repository. A later episode describes the Markdown we use.

Teaching Tools

We do not prescribe what tools instructors should use when actually teaching: the Jupyter Notebook, RStudio, and the good ol’ command line are equally welcome up on stage. All we specify is the format of the lesson notes.

Jekyll

GitHub uses Jekyll to turn Markdown into HTML. It looks for text files that begin with a header formatted like this:

---
variable: value
other_variable: other_value
---
...stuff in the page...

and inserts the values of those variables into the page when formatting it. The three dashes that start the header must be the first three characters in the file: even a single space before them will make Jekyll ignore the file.

The header’s content must be formatted as YAML, and may contain Booleans, numbers, character strings, lists, and dictionaries of name/value pairs. Values from the header are referred to in the page as page.variable. For example, this page:

---
name: Science
---
Today we are going to study {{page.name}}.

is translated into:

<html>
  <body>
    <p>Today we are going to study Science.</p>
  </body>
</html>

Back in the Day…

The previous version of our template did not rely on Jekyll, but instead required authors to build HTML on their desktops and commit that to the lesson repository’s gh-pages branch. This allowed us to use whatever mix of tools we wanted for creating HTML (e.g., Pandoc), but complicated the common case for the sake of uncommon cases, and didn’t model the workflow we want learners to use.

Configuration

Jekyll also reads values from a configuration file called _config.yml, which are referred to in pages as site.variable. The lesson template does not include _config.yml, since each lesson will change some of its value, which would result in merge collisions each time the lesson was updated from the template. Instead, the template contains a script called bin/lesson_initialize.py which should be run once to create an initial _config.yml file (and a few other files as well). The author should then edit the values in the top half of the file.

Collections

If several Markdown files are stored in a directory whose name begins with an underscore, Jekyll creates a collection for them. We rely on this for both lesson episodes (stored in _episodes) and extra files (stored in _extras). For example, putting the extra files in _extras allows us to populate the “Extras” menu pulldown automatically. To clarify what will appear where, we store files that appear directly in the navigation bar in the root directory of the lesson. The next episode describes these files.

Key Points

  • Lessons are stored in Git repositories on GitHub.

  • Lessons are written in Markdown.

  • Jekyll translates the files in the gh-pages branch into HTML for viewing.

  • The site’s configuration is stored in _config.yml.

  • Each page’s configuration is stored at the top of that page.

  • Groups of files are stored in collection directories whose names begin with an underscore.


Lesson Organization

Overview

Teaching: 10 min
Exercises: 0 min
Questions
  • How are the files in a lesson organized?

Objectives
  • Explain overall organization of lesson files.

Each lesson is made up of episodes, which are focused on a particular topic and include time for both teaching and exercises. The episodes of this lesson explain the tools we use to create lessons and the formatting rules those lessons must follow.

Why “Episodes”?

We call the parts of lessons “episodes” because every other term (like “topic”) already has multiple meanings, and because it encourages us to think of breaking up our lessons into chunks that are about as long as a typical movie scene, which is better for learning than long blocks without interruption.

Our lessons need artwork, CSS style files, and a few bits of Javascript. We could load these from the web, but that would make offline authoring difficult. Instead, each lesson’s repository is self-contained.

The diagram below shows how source files and directories are laid out, and how they are mapped to destination files and directories:

Source and Destination Files

Collections

As described earlier, files that appear as top-level items in the navigation menu are stored in the root directory. Files that appear under the “extras” menu are stored in the _extras directory, while lesson episodes are stored in the _episodes directory.

Helper Files

As is standard with Jekyll sites, page layouts are stored in _layouts and snippets of HTML included by these layouts are stored in _includes. Each of these files includes a comment explaining its purpose.

Authors do not have to specify that episodes use the episode.html layout, since that is set by the configuration file. Pages that authors create should have the page layout unless specified otherwise below.

The assets directory contains the CSS, Javascript, fonts, and image files used in the generated website. Authors should not modify these.

Standard Files

When the lesson repository is first created, the initial author should create a README.md file containing a one-line explanation of the lesson’s purpose.

The lesson template provides the following files which should not be modified:

Starter Files

The bin/lesson_initialize.py script creates files that need to be customized for each lesson:

CONTRIBUTING.md
Contribution guidelines. The issues and repo links at the bottom of the file must be changed to match the URLs of the lesson: look for uses of FIXME.
_config.yml
The Jekyll configuration file. This must be edited so that its links and other settings are correct for this lesson.
  • carpentry should be either “dc” (for Data Carpentry), “lc” (for Library Carpentry), “swc” (for Software Carpentry), “cp” (for Carpentries), or “incubator” (for lessons in The Carpentries Incubator).
  • title is the title of your lesson, e.g., “Defence Against the Dark Arts”.
  • email is the contact email address for the lesson.
CITATION
A plain text file explaining how to cite this lesson.
AUTHORS
A plain text file listing the names of the lesson’s authors.
index.md
The home page for the lesson.
  1. It must use the lesson layout.
  2. It must not have a title field in its YAML header.
  3. It must open with a few paragraphs of explanatory text.
  4. That introduction must be followed by a single .prereq blockquote detailing the lesson’s prerequisites. (Setup instructions appear separately.)
  5. That must be followed by inclusion of syllabus.html, which generates the syllabus for the lesson from the metadata in its episodes.
reference.md
A reference guide for the lesson. The template will automatically generate a summary of the episodes’ key points.
  1. It must use the reference layout.
  2. Its title must be "Reference".
  3. It should include a glossary, laid out as a description list.
  4. It may include other material as appropriate.
setup.md
Detailed setup instructions for the lesson. Note that we usually divide setup instructions by platform, e.g., include level-2 headings for Windows, macOS, and Linux with instructions for each. The workshop template links to the setup instructions for core lessons.
  1. Its title must be "Setup"..
  2. It should include whatever setup instructions are required.
_extras/about.md
General notes about this lesson. This page includes brief descriptions of The Carpentries, and is a good place to put institutional acknowledgments.
_extras/discussion.md
General discussion of the lesson contents for learners who wish to know more: This page normally includes links to further reading and/or brief discussion of more advanced topics.
  1. Its title must be "Discussion".
  2. It may include whatever content the author thinks appropriate.
_extra/figures.md and _includes/all_figures.html
Does nothing but include _includes/all_figures.html, which is (re)generated by make lesson-figures. This page displays all the images referenced by all of the episodes, in order, so that instructors can scroll through them while teaching.
_extras/guide.md
The instructors’ guide for the lesson. This page records tips and warnings from people who have taught the lesson.
  1. Its title must be "Instructors' Guide".
  2. It may include whatever content the author thinks appropriate.

Figures

All figures related with the lesson must be placed inside the directory fig at the root of the project.

Episode Order

By default, the order of Episodes and Extras in the lesson is determined by the name of the file, e.g. an episode with the filename 01-introduction.md appears before a lesson with the filename 02-cheese-rolling.md. To override this default ordering, define the episode_order and/or extras_order parameters in _config.yml. The value of these parameters should be an array of filenames with the extension removed, e.g.

episode_order:
    - introduction
    - first_steps
    - running
    - jumping
    - gymnastics
    - olympics
    - conclusion

Some lesson developers have found this useful when using the lesson template to create more modular material and/or when working with material in the early stages of development that is likely to be added to and reorganised. Any files in the relevant collection folder (e.g. _episodes) not included in the array will be omitted from the lesson navigation (but will still be built).

Key Points

  • Auxiliary files are stored in the _layouts, _includes, and assets directories.

  • The code of conduct, license, Makefile, and contribution guidelines should not be modified.

  • The README, authors’ list, and citation instructions must be updated for each lesson.

  • The home page, reference guide, setup instructions, discussion page, and instructors’ guide must be updated for each lesson.

  • The Makefile stores commonly-used commands.


Formatting

Overview

Teaching: 10 min
Exercises: 0 min
Questions
  • How are The Carpentries lessons formatted?

Objectives
  • Explain the header of each episode.

  • Explain the overall structure of each episode.

  • Explain why blockquotes are used to format parts of episodes.

  • Explain the use of code blocks in episodes.

A lesson consists of one or more episodes, each of which has:

The diagram below shows the internal structure of a single episode file (click on the image to see a larger version):

Formatting Rules

Maximum Line Length

Limit all lines to a maximum of 100 characters. bin/lesson_check.py will report lines longer than 100 characters and this can block your contributions of being accepted.

The two reasons behind the decision to enforce a maximum line length are (1) make diff and merge easier in the command line and other user interfaces and (2) make update of translation of the lessons easier.

Locations and Names

Episode files are stored in _episodes or, for the case of R Markdown files, _episodes_rmd so that Jekyll will create a collection for them. Episodes are named dd-subject.md, where dd is a two-digit sequence number (with a leading 0) and subject is a one- or two-word identifier. For example, the first three episodes of this example lesson are _episodes/01-design.md, _episodes/02-tooling.md and _episodes/03-formatting.md. These become /01-design/index.html, /02-tooling/index.html, and /03-formatting/index.html in the published site. When referring to other episodes, use:

[link text]({{ page.root }}{% link _episodes/dd-subject.md %})

i.e., use Jekyll’s link tag and the name of the file.

Episode Header

Each episode’s YAML header must contain:

These values are stored in the header so that Jekyll will read them and make them accessible in other pages as site.episodes.the_episode.key, where the_episode is the particular episode and key is the key in the YAML header. This lets us do things like list each episode’s key questions in the syllabus on the lesson home page.

Episode Structure

The episode layout template in _layouts/episode.html automatically creates an introductory block that summarizes the lesson’s teaching time, exercise time, key questions, and objectives. It also automatically creates a closing block that lists its key points. In between, authors should use only:

Authors should not use:

Linking section IDs

In the HTML output each header of a section, code sample, exercise will be associated with an unique ID (the rules of the ID generation are given in kramdown documentation, but it is easier to look for them directly in the page sources). These IDs can be used to easily link to the section by attaching the hash (#) followed by the ID to the page’s URL (like this). For example, the instructor might copy the link to the etherpad, so that the lesson opens in learners’ web browser directly at the right spot.

Formatting Code

Inline code fragments are formatted using back-quotes. Longer code blocks are formatted by opening and closing the block with ~~~ (three tildes), with a class specifier after the block:

~~~
for thing in collection:
    do_something
~~~
{: .source}

which is rendered as:

for thing in collection:
    do_something

The class specified at the bottom using an opening curly brace and colon, the class identifier with a leading dot, and a closing curly brace. The template provides three styles for code blocks:

.source: program source.
.output: program output.
.error: error messages.

Syntax Highlighting

The following styles like .source, but include syntax highlighting for the specified language. Please use them where possible to indicate the type of source being displayed, and to make code easier to read.

.language-bash: Bash shell commands:

echo "Hello World"

.language-html: HTML source:

<html>
<body>
<em>Hello World</em>
</body>
</html>

.language-make: Makefiles:

all:
    g++ main.cpp hello.cpp -o hello

.language-matlab: MATLAB source:

disp('Hello, world!')

.language-python: Python source:

print("Hello World")

.language-r: R source:

cat("Hello World")

.language-sql: SQL source:

CREATE PROCEDURE HelloWorld AS
PRINT 'Hello, world!'
RETURN (0)

Highlighting for other languages

You may use other language-* classes to activate syntax highlighting for other languages. For example,

~~~
title: "YAML Highlighting Example"
description: "This is an example of syntax highlighting for YAML."
array_values:
    - value_1
    - value_2
~~~
{: .language-yaml }

will produce this:

title: "YAML Highlighting Example"
description: "This is an example of syntax highlighting for YAML."
array_values:
    - value_1
    - value_2

Note that using .language-* classes other than .language-bash .language-html, .language-make, .language-matlab, .language-python, .language-r, or .language-sql will currently cause one of the tests in the lesson template’s make lesson-check to fail for your lesson, but will not prevent lesson pages from building and rendering correctly.

Special Blockquotes

We use blockquotes to group headings and text rather than wrapping them in div elements. in order to avoid confusing Jekyll’s parser (which sometimes has trouble with Markdown inside HTML). Each special blockquote must started with a level-2 header, but may contain anything after that. For example, a callout is formatted like this:

> ## Callout Title
>
> text
> text
> text
>
> ~~~
> code
> ~~~
> {: .source}
{: .callout}

(Note the empty lines within the blockquote after the title and before the code block.) This is rendered as:

Callout Title

text text text

code

The lesson template defines styles for the following special blockquotes:

.callout

An aside or other comment.

.challenge

An exercise.

.checklist

Checklists.

.discussion

Discussion questions.

.keypoints

Key points of an episode.

.objectives

Episode objectives.

.prereq

Prerequisites.

.solution

Exercise solution.

.testimonial

A laudatory quote from a user.

Note that .challenge and .discussion have the same color but different icons. Note also that one other class, .quotation, is used to mark actual quotations (the original purpose of the blockquote element). This does not add any styling, but is used to prevent the checking tools from complaining about a missing class.

Most authors will only use .callout, .challenge, and .prereq, as the others are automatically generated by the template. Note that .prereq is meant for describing things that learners should know before starting this lesson; setup instructions do not have a particular style, but are instead put on the setup.md page.

Note also that solutions are nested inside exercises as shown below:

> ## Challenge Title
>
> This is the body of the challenge.
>
> ~~~
> it may include some code
> ~~~
> {: .source}
>
> > ## Solution
> >
> > This is the body of the solution.
> >
> > ~~~
> > it may also include some code
> > ~~~
> > {: .output}
> {: .solution}
{: .challenge}

The double indentation is annoying to edit, but the alternatives we considered and discarded are worse:

  1. Use HTML <div> elements for the challenges. Most people dislike mixing HTML and Markdown, and experience shows that it’s all too easy to confuse Jekyll’s Markdown parser.

  2. Put solutions immediately after challenges rather than inside them. This is simpler to edit, but clutters up the page and makes it harder for tools to tell which solutions belong to which exercises.

Key Points

  • Lesson episodes are stored in _episodes/dd-subject.md.

  • Each episode’s title must include a title, time estimates, motivating questions, lesson objectives, and key points.

  • Episodes should not use sub-titles or HTML layout.

  • Code blocks can have the source, regular output, or error class.

  • Special sections are formatted as blockquotes that open with a level-2 header and close with a class identifier.

  • Special sections may be callouts or challenges; other styles are used by the template itself.


Using RMarkdown

Overview

Teaching: 10 min
Exercises: 2 min
Questions
  • How to write a lesson using RMarkdown?

Objectives
  • Explain how to use RMarkdown with the new lesson template.

  • Demonstrate how to include pieces of code, figures, and challenges.

This episode demonstrates all the features that can be used when writing a lesson in RMarkdown.

To generate the site, you will need to have the following packages installed:

install.packages(c("knitr", "stringr", "checkpoint"))

If the lesson uses additional packages, the script that converts the Rmd files into markdown, will detect them and install them for you, when you run make serve or make site.

This first chunk is really important, and need to be included at the beginning of each episode written in RMarkdown. The first line ensures that the chunks have the correct styling in the rendered website. The second line is optional but allows you to ensure that figures that are generated by the lessons will have the same prefix for each episode (you should use a different prefix for each episode using the episode number for instance).

```{r, include=FALSE}
source("../bin/chunk-options.R")
knitr_fig_path("05-")
```

The rest of the lesson should be written as a normal RMarkdown file. You can include chunk for codes, just like you’d normally do.

Normal output:

1 + 1
[1] 2

Output with error message:

x[10]
Error in eval(expr, envir, enclos): object 'x' not found

Output generating figures:

library(ggplot2)
ggplot(diamonds, aes(x = carat,  y = price, color = cut)) +
    geom_point()

plot of chunk plot-example

For the challenges and their solutions, you need to pay attention to where the > go and where to leave blank lines. You can include code chunks in both the instructions and solutions. For instance this:

> ## Challenge: Can you do it?
>
> What is the output of this command?
>
>
> ~~~
> paste("This", "new", "template", "looks", "good")
> ~~~
> {: .language-r}
>
> > ## Solution
> >
> >
> > ~~~
> > [1] "This new template looks good"
> > ~~~
> > {: .output}
> {: .solution}
{: .challenge}

will generate this:

Challenge: Can you do it?

What is the output of this command?

paste("This", "new", "template", "looks", "good")

Solution

[1] "This new template looks good"

Key Points

  • Edit the .Rmd files not the .md files

  • Run make serve to knit documents and preview lesson website locally


Style Guide

Overview

Teaching: 10 min
Exercises: 0 min
Questions
  • How are keyboard key combinations written?

Objectives
  • Explain keyboard key combination.

Emphasis

Markdown treats asterisks (*) indicators of emphasis, and renders text marked up like this in italics. We use emphasis

Strong Emphasis

Markdown treats double asterisks (**) as indicators of strong emphasis, and renders text marked up like this in boldface. We use strong emphasis

Span of Code

Markdown treats backtick quotes (`) as indicators of a span of code. We use span of code

Keyboard Key

When making reference to a keyboard key that the reader should press the HTML tag <kbd> must be used to enclose the key label. For example, “to delete the cell press D” should be write as

to delete the cell press <kbd>D</kbd>

The table below covers most of the keyboard key labels.

Keyboard key Style Note Example
Letters Always capital. <kbd>A</kbd>
Numbers   <kbd>1</kbd>
Punctuation mark   <kbd>*</kbd>
Function Capital F followed by the number. <kbd>F12</kbd>
Alt Only first letter capital. <kbd>Alt</kbd>
Backspace Only first letter capital. <kbd>Backspace</kbd>
Command Only first letter capital. <kbd>Command</kbd>
Ctrl Only first letter capital. <kbd>Ctrl</kbd>
Delete Only first letter capital. <kbd>Delete</kbd>
End Only first letter capital. <kbd>End</kbd>
Esc Only first letter capital. <kbd>Esc</kbd>
Home Only first letter capital. <kbd>Home</kbd>
Insert Only first letter capital. <kbd>Insert</kbd>
Page Down Use “PgDn”. <kbd>PgDn</kbd>
Page Up Use “PgUp”. <kbd>PgUp</kbd>
Print Screen Use “PrtScr”. <kbd>PrtScr</kbd>
Return Only first letter capital. We use “Return” instead of “Enter”. <kbd>Return</kbd>
Shift Only first letter capital. <kbd>Shift</kbd>
Spacebar Only first letter capital. <kbd>Spacebar</kbd>
Tab Only first letter capital. <kbd>Tab</kbd>
Down arrow Use Unicode “Downwards arrow” (8595). <kbd>↓</kbd>
Left arrow Use Unicode “Leftwards arrow” (8592). <kbd>←</kbd>
Right arrow Use Unicode “Rightwards arrow” (8594). <kbd>→</kbd>
Up arrow Use Unicode “Upwards arrow” (8593). <kbd>↑</kbd>

Keyboard Key Combination

When making reference to a keyboard key combination that the reader should press, insert a plus sign without space between each one of the keys. For example, “press Ctrl+X to quit nano” should be written as:

press <kbd>Ctrl</kbd>+<kbd>X</kbd> to quit nano.

Please label links with meaningful texts, in order to improve accessibility. Please avoid click here or similar.

Use double quotes for menu and submenu items. For multiple menu and submenu options in a sequence, use the right angle bracket or greater than sign.

e.g.:

“Help” > “Check for updates”

Title Casing

Lesson and episode titles should be written in title case e.g.

Saving the World, One Commit at a Time

An exception should be made where the title includes the name of a tool/library/command that must be typed in lower case when used, e.g.

Data Visualisation with matplotlib

(In HTML files, such as the workshop schedule and syllabus files in the workshop webpage template, tool/command/library names such as matplotlib in the example above can be correctly formatted using <code></code> tags.)

Key Points

  • Keyboard keys need to use <kbd> HTML tag.


Checking and Previewing

Overview

Teaching: 5 min
Exercises: 0 min
Questions
  • How can lesson formatting be checked?

  • How can lessons be previewed?

Objectives
  • Run the lesson checking script and interpret its output correctly.

  • Preview a lesson locally.

The lesson template comes with several utilities to simplify lesson development and maintenance.

Checking

The template includes a Python program to check whether lesson files conform to our template. You can run this using make lesson-check, which in turn invokes bin/markdown_ast.rb to parse Markdown files and bin/lesson_check.py to check their structure. The former is written in Ruby, and uses Jekyll’s own Markdown parser (called Kramdown) so that we are guaranteed to be checking the same dialect of Markdown that Jekyll uses on GitHub. The latter is written in Python 3, and executes all of the checks.

The template also includes bin/repo_check.py, which can be invoked by running make repo-check. This program looks in _config.yml to find the repository’s URL, then checks that the repository has the right labels set up for issues and pull requests. Other checks will be added as time goes by.

Previewing

Jekyll can be used in two ways: to compile source files into HTML pages in the _site directory, or to do that and also run a small web server at http://0.0.0.0:4000/ so that the pages can be previewed. We recommend using the latter, since it gives a more accurate impression of what your lesson will actually look like.

The Makefile in the root directory of the project contains commands for building the site. make site builds files but does not run a server, while make serve builds the files and runs a server. (It also re-builds the site whenever it notices changes in the source files.) Run make on its own to get a full list of commands.

In order to use Jekyll and/or the checking script, you may need to install it and some other software. The setup instructions explain what you need and how to get it.

Key Points

  • Lessons are checked by running make lesson-check.

  • The checker uses the same Markdown parser as Jekyll.

  • Lessons can be previewed by running make serve.


Coffee Break

Overview

Teaching: min
Exercises: min
Questions
Objectives

Use the break layout for placeholder episodes representing coffee breaks and lunches. These episodes do not have exercises, questions, objectives, or keypoints in their metadata, but must have a “break” field to show how long the break is (in minutes).

Key Points