Summary and Schedule
The following documentation summarises the steps needed to get started using the Workbench tools.
| Setup Instructions | Download files required for the lesson | |
| Duration: 00h 00m | 1. Building Lessons |
How do we build lessons using a local Workbench installation? How do we build lessons using the online GitHub architecture? |
| Duration: 00h 10m | Finish |
The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.
Installation of the Workbench
The Workbench lesson infrastructure is built on Git, the R language, and pandoc. It consists of four components:
- The source content, i.e. the “lesson” (plain markdown or RMarkdown files organized into folders with a configuration yaml file)
- The engine (R package {sandpaper}) to process the lesson content markdown to HTML
- The validator (R package {pegboard}) to parse the source markdown files and highlight common errors
- The renderer (R package {varnish}) to generate the final HTML, CSS, and JavaScript styling elements for the lesson website
Details of how these tools work together are explained in the TODO. In short, you can interact with the lesson source content and the Workbench tools to author and preview your lesson.
What installation method should I choose?
We provide four mechanisms to interact with the Workbench:
- Devcontainers (online or local) [recommended]
- We recommend using devcontainers as everything will be installed for you whether you are using an online or local environment, with minimal setup.
- Lessons with devcontainer support will have a
.devcontainerfolder in the root of the repository. - If no devcontainer is available for the lesson you wish to work on, please use the manual installation route, or Docker if you are comfortable with it.
- Docker (local)
- The Docker option can be useful for those that already have experience with Docker and containers, and are happy to do some manual configuration or wish to have more flexibility.
- Initial setup with docker is very straightfoward but can get complicated depending on your operating system!
- Manual local installation (local)
- Installing all the requirements manually on your local operating system is a longer process, but can be useful if you want to learn how to get an environment set up and will teach you more about the Workbench framework.
- If Docker and devcontainers are not suitable for you, installing things manually is a great option and still relatively quick to do!
- GitHub (online)
- Moving your development completely online by using GitHub’s Codespaces or Dev environment is an excellent choice if you have fast internet and are comfortable with this process.
- Be advised that this option may incur a cost depending on your GitHub account status!
Why would I want to install locally if all these other options exist?
You can use the Workbench completely within devcontainers or Docker locally, or through GitHub using our templates.
However, we still would recommend working through the local setup instructions at least once as they can save you a lot of time during lesson development and debugging in future.
Devcontainers
Devcontainers are a way of remotely or locally setting up a coding environment based on a preconfigured “blueprint”. This blueprint is used by Integrated Development Environments (IDEs) like VSCode, vscodium, IDEA, or Zed, to initiate and configure an environment for you. This can make it much quicker to get up and running with lesson development.
To use devcontainers you need:
- Docker: please read the Docker installation instructions to install Docker Desktop or the Docker Engine for your operating system.
- An IDE that supports devcontainers: We recommend VSCode as it has the most complete and easy to set up devcontainer support. We acknowledge that VSCode is free but not completely open source.
Please note, if you are not comfortable with using Docker or the Workbench, we recommend going through the manual installation steps below.
Starting up the devcontainer
Make sure Docker Desktop is running and that you are connected to the Internet.
Open VSCode:
- either, by double clicking the icon in your OS, going to
File,Open Folder..., and navigating to a lesson folder - or, open a terminal, navigate to a lesson folder and start VSCode by typing:
NB: Note the space between code and .!
Once VSCode has opened the folder, after a short time, a pop-up
should appear in the bottom right of the IDE saying
Folder contains a Devcontainer configuration file. Reopen folder to develop in a container,
with an option button to Reopen in container. Click this
button and wait for VSCode to read the configuration file and setup the
devcontainer environment.
Troubleshooting
If this pop-up does not appear, open the command menu in VSCode with
the F1 function key. Your cursor will appear in the command
menu bar at the top of the window, prefixed with a >
symbol. Start typing open folder and select the
Dev Containers: Open Folder in Container... option.
Similarly if you have any issues with the container at any point, you
can open the command menu with F1, start typing
rebuild, and select the
Dev Containers: Rebuild and Reopen in Container.... This
should be used judiciously as it will often reinstall dependencies so
can take a few minutes.
Various pop-ups will appear in the bottom right, showing progress.
You can click on the Show log link in any of the popups
that show it to see any logs of the setup process.
The setup process can take a few minutes the first time you run it for a lesson. After the first time, it will be much quicker to start subsequent uses of the environment as the Docker image would be downloaded and stored for reuse.
Lessons with a lot of dependencies
For those R Markdown lessons that require a large number of R packages, this first start of the devcontainer can take quite a long time.
Running the Workbench
Open a terminal in VSCode by going to the Terminal menu
in the top menu bar and click New Terminal. An Ubuntu
bash terminal window should appear in VSCode. This bash
shell will be running inside the devcontainer, so
should look something like:
Here, you can perform any shell or R actions as you would usually for lesson development.
For more information, please check the full devcontainer documentation.
What if I don’t want to use VSCode?
Sadly the devcontainer specification is only fully supported in a
small number of IDEs. VSCode is recommended in our lessons and it
natively supports devcontainers. Getting devcontainers to work in other
fully open source IDEs like vscodium and zed
are not as simple.
Therefore we would recommend using VSCode as whilst it is partially open source, it is a reliable and very well supported piece of software.
If this is not an option, then we recommend using the Docker image directly as it comes bundled with RStudio which you can use to work on your lesson.
Sneaky extra: If you want to run RStudio Server from VSCode, in a terminal shell running in the VSCode devcontainer, type:
This will start the RStudio Server within the devcontainer, and you
can then access RStudio by opening a web browser and going to
http://localhost:8787. Fancy!
Are devcontainers available for all lessons?
Devcontainer blueprints are already provided by our workbench-template-rmd and workbench-template-md lesson templates, so when creating a new lesson you will get access to devcontainers automatically.
However, not all Workbench-compatible lessons will have devcontainers
enabled (a .devcontainer folder within the repository). To
add devcontainer support to a pre-existing lesson, please follow the devcontainer import instructions TODO
We will also be rolling these out into existing lessons across our official lesson programs too!
Docker
The documentation below for each operating system goes through how to manually install the Workbench dependencies, requirements, and packages. To make this lengthy process shorter, the Workbench development team also provides Docker images of each release of the Workbench that are used for our GitHub workflows that build our core curriculum lessons. You can use these Docker images to run the Workbench on your local system too!
Docker Desktop is available for all operating systems (Windows, Mac and Linux), so we recommend installing this.
Docker Desktop
Install Docker Desktop for your operating system.
Once installed, choose one of the following ways to interact with the Docker system:
- Via the Docker Desktop GUI app
- Via the command line terminal (bash on Linux/Windows WSL/git bash, terminal on the Mac)
Running the Workbench Docker container
After Docker Desktop/Docker Engine is set up, you can use the
following command from the terminal (git bash, WSL, or
MacOS Terminal app):
BASH
# go home
cd ~
# make a `lessons` folder in your home directory and clone in a lesson
mkdir ~/lessons
cd ~/lessons
git clone git@github.com:swcarpentry/shell-novice.git
# create a workbench-lessons named volume, and copy in the shell-novice content
curl -s https://raw.githubusercontent.com/carpentries/workbench-docker/main/scripts/setup_named_volume.sh | bash -s -- ~/lessons/shell-novice
# start the workbench container
curl -s https://raw.githubusercontent.com/carpentries/workbench-docker/main/scripts/run_workbench.sh | bashOr with docker run directly, if you want to control
various docker options:
BASH
docker run --name carpentries-workbench \
-p 8787:8787 \
-v ~/.ssh:/home/rstudio/.ssh:ro \
-v ~/.gitconfig:/home/rstudio/.gitconfig \
-e DISABLE_AUTH=true \
-d carpentries/workbench-docker:latestIn the Docker Desktop app, you should see an entry appear in the
“Containers” tab with the name carpentries-workbench.
You can now open a web browser and navigate to
http://localhost:8787 which will connect to the RStudio
server running in the Docker container. You’re good to go! Please
continue with the Test Your Installation
instructions.
More information and docker runtime options are described in the Docker guide and Workbench developer docker documentation.
Please note, if you are not comfortable with Docker or the Workbench, we recommend going through the manual installation steps below.
Manual Local Installation
Required Software
This setup document will take you through the process of installing or upgrading the required software on your local computer or server.
Git (≥ 2.28 recommended)
R (≥ 4.x)
pandoc (≥ 3.x)
-
The lesson infrastructure R packages
- {sandpaper} (development version)
- {varnish} (development version)
- {pegboard} (development version)
- {tinkr} (markdown parser required by {pegboard})
Once you have Git, R, and Pandoc installed, these packages can be installed and updated via:
R
install.packages(c("sandpaper", "varnish", "pegboard"), repos = c("https://carpentries.r-universe.dev/", getOption("repos")))
Recommended Software
If you are using R or pandoc for the first time, we recommend using the RStudio IDE for the following reasons:
- It comes with pandoc pre-installed.
- It works consistently across all major platforms.
- It provides a dedicated
bashterminal so you can easily switch between R and Git operations. - There are convenient keyboard shortcuts to preview lessons.
- On Windows, it will automatically detect your R installation without
you needing to edit your
PATH.
If you do not want to use RStudio, that’s perfectly okay and expected! We want to you to be able work with the new template with the tools with which you’re most familiar. If you feel comfortable using a different tool (e.g. the command line or VSCode), then you should install R and pandoc separately and make sure that they are in your PATH.
If you already have software installed and are curious if you should update it to a newer version, the answer is almost always, yes. Newer versions will often contain important bug fixes that are important to the security of your computer.
Jump to the installation instructions for your system: Windows, MacOS, or Linux.
Installing on Windows
Git
We recommend installing git via the Git for Windows installer. The installer is going to ask a lot of questions, so we recommend following The Carpentries instructions for workshop participants.
Test your installation
To test that you have git installed, open your command line by
pressing Windows+R and type cmd to
bring up the command prompt. From there, you can type
git --version to see the version of your git installation.
You might see something like this:
OUTPUT
git version 2.31.1.windows.1If, however, you see this error, then you should try to install git again.
ERROR
'git' is not recognised as an internal or external command, operable program or batch file.R
Install the latest version of R for Windows. There is also a video tutorial up on The Carpentries instructions for workshop participants that can be quite helpful for parsing the steps of installing R on Windows.
Optional: Want to add R to your PATH?
As we mention above, we recommend using RStudio
for your lesson, but if you want to be able to integrate the lesson
infrastructure into your own preferred workflow, you need to have R on
your path. The catch is that R for Windows does not automatically set
your PATH variable.
Here are
some instructions on setting up your PATH on Windows using both the GUI
and CLI. Note that R will normally install at something like
C:\Program Files\R\R-4.4.0\bin\x64, but if you are not an
administrator on your device, it will install R in your
My Documents folder.
To verify that R is installed in your PATH, open your
command line by pressing Windows+R and type
cmd to bring up the command prompt. From there, you can
type R --version at the prompt. Your output should be
similar to below, with a version ≥ 4.x.
OUTPUT
R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-mingw32/x64 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.pandoc
There are two ways to install pandoc:
Via RStudio (recommended)
Since pandoc comes bundled with RStudio, you can install it by installing the latest version of RStudio. You can download the installer from the RStudio website.
Via the pandoc website
If you are comfortable adding R to your windows PATH (see previous section), then you can install pandoc by using the download provided on its website at https://pandoc.org/installing.html
Infrastructure R packages
To install the R packages, you will need to open RStudio (or start R from the command line if you did not install RStudio) and enter the following lines into the console.
R
# register the repositories for The Carpentries and CRAN
options(repos = c(
carpentries = "https://carpentries.r-universe.dev/",
CRAN = "https://cran.rstudio.com/"
))
# Install the template packages to your R library
install.packages(c("sandpaper", "varnish", "pegboard"))
Installing on MacOS
Git
You should have git pre-installed on your macOS, but it is likely that this is an old version. We recommend installing The latest version of Git for MacOS. For a video guide, you can look at the instructions for workshop participants.
Test your installation
To test your installation of Git and confirm it works, open Terminal.app and type the following:
OUTPUT
git version 2.31.0If you have the default version of git, you might see this output, and that’s okay for the purposes of this template.
OUTPUT
git version 2.24.3 (Apple Git-128)R
Homebrew Not Recommended
Installing R via Homebrew can allow you to customise your
installation, but you lose the advantage of having readily available
package binaries. Moreover, if you do not have the required C libraries
(e.g. libxslt) installed, then the installation of some
packages will fail. So, unless you are prepared for this, please
do not use Homebrew to install R.
You can install the latest R release for MacOS from https://cran.r-project.org/bin/macosx. There is also a video tutorial up on The Carpentries instructions for workshop participants that can be quite helpful for parsing the steps of installing R on MacOS.
Test your installation
You can test your installation of R by opening
Terminal.app and typing R --version into
the prompt. Your output should be similar to below, with a version ≥
4.x:
OUTPUT
R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.pandoc
There are two ways to install pandoc:
Via RStudio (recommended)
Since pandoc comes bundled with RStudio, you can install it by
installing the latest version of RStudio. You can download the installer from the RStudio
website. RStudio will be a *.dmg (disk image) that you
will double click to open a window that will look something like
this:
You should drag the RStudio icon to the left, into the Applications folder to install RStudio on your computer.
Via the pandoc website
If are more comfortable using R from the command line, then you can
install pandoc by
clicking the “Download the latest installer for macOS” button. This will
save a file called pandoc-X.XX-macOS.pkg installer to your
computer. Open the installer and follow the instructions to install
pandoc on your computer.
Infrastructure R packages
To install the R packages, you will need to open RStudio (or start R from the command line if you did not install RStudio) and enter the following lines into the console.
R
# register the repositories for The Carpentries and CRAN
options(repos = c(
carpentries = "https://carpentries.r-universe.dev/",
CRAN = "https://cran.rstudio.com/"
))
# Install the template packages to your R library
install.packages(c("sandpaper", "varnish", "pegboard"))
Installing on Linux
Instructions for installing on Linux are nuanced due to the variety
and availability of libraries and dependencies for each distribution,
e.g. Ubuntu is Debian based whereas Fedora is Red Hat based. These
instructions will use Ubuntu as the preferred distribution. The default
apt repository is often out of date, so you will need to
use a Personal Package Archive
aka PPA to install the latest version of a particular software,
which will be included in these instructions.
Git
Many distributions include git by default, but it is often outdated. It is useful to try to update in case a newer version is available:
R
To install R, you can visit CRAN’s Linux page to check if your platform is supported. Detailed instructions exist for Ubuntu. Here are the commands to register the PPA on your machine and then install R:
BASH
# update indices
sudo apt update -qq
# install two helper packages we need
sudo apt install --no-install-recommends software-properties-common dirmngr
# import the signing key (by Michael Rutter) for these repo
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
# add the R 4.0 repo from CRAN -- adjust 'focal' to 'groovy' or 'bionic' as needed
sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
# Install R
sudo apt install --no-install-recommends r-baseTest your installation
Test your R installation by opening your terminal and running
R --version into the prompt. Your output should be similar
to below, with a version ≥ 4.x:
OUTPUT
R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.
pandoc
There are two ways to install pandoc:
Via RStudio (recommended)
Pandoc comes bundled with RStudio. You can download the installer from the RStudio website. When installing RStudio for Linux, your distribution may not be shown on the landing page. If your Ubuntu version is not listed, check the RStudio Previous Versions page to see if there are builds available.
Optional: verify the install
You can optionally verify the download before installing by following the instructions at https://posit.co/code-signing/.
Via the pandoc website
If you are more comfortable using R from the command line, then you
can install pandoc from the releases page. From the
list on the GitHub page, for Ubuntu, choose the appropriate
.deb file for your chip architecture (i.e. AMD64 or
ARM64).
Infrastructure R packages
Linux packages normally need to be compiled into binaries by your system, which can take a long time the first time it happens. RStudio provides a package manager that pre-compiles Linux binaries. Note that you do not have to be using RStudio to take advantage of these binaries. The one we are using is set up for the current Long Term Support (LTS) version of Ubuntu - 24.04 (noble).
Dependencies of Dependencies
If you are not used to installing software on Linux, it can be frustrating sometimes because things can go wrong and it’s not always immediately clear why. The same is true for some R packages with compiled code.
Some packages require underlying C libraries (e.g. the xml2 library),
which are catalogued for Ubuntu in The Carpentries R
Universe and available via the
API. To produce a list (you may need to
sudo apt install jq if it is not already on your
system):
BASH
curl https://carpentries.r-universe.dev/stats/sysdeps 2> /dev/null | jq -r '.headers[0] | select(. != null)'This list can be sent to apt install to install
everything:
BASH
sudo apt-get install -y \
$(curl https://carpentries.r-universe.dev/stats/sysdeps 2> /dev/null | jq -r '.headers[0] | select(. != null)') 2> /dev/null \
|| echo "Not on Ubuntu"After you have these installed, you will be able to install the required R packages without error.
What if I have a different version of Linux?
For the dependencies above, you can browse the rstudio/r-system-requirements repository to find the correct formulation for your computer.
In addition, you should check the supported operating systems for the Posit Package Manager to see if you will benefit from pre-built binaries.
To install the R packages, you will need to open RStudio (or start R from the command line if you did not install RStudio) and enter the following lines into the console:
R
# Set the default HTTP user agent to get pre-built binary packages
RV <- getRversion()
OS <- paste(RV, R.version["platform"], R.version["arch"], R.version["os"])
codename <- sub("Codename.\t", "", system2("lsb_release", "-c", stdout = TRUE))
options(HTTPUserAgent = sprintf("R/%s R (%s)", RV, OS))
# register the repositories for The Carpentries and CRAN
options(repos = c(
carpentries = "https://carpentries.r-universe.dev/",
CRAN = paste0("https://packagemanager.posit.co/all/__linux__/", codename, "/latest")
))
# Install the template packages to your R library
install.packages(c("sandpaper", "varnish", "pegboard"))
Saving these settings for later
To not have to run this block of code every time you want to update,
add the following code into your ~/.Rprofile to run it
every time you open your terminal:
Open the nano text editor:
And copy and paste the following code at the bottom of the file. Once you have done so, type Ctrl+X to save and exit.
R
local({
# Set the default HTTP user agent to get pre-built binary packages
RV <- getRversion()
OS <- paste(RV, R.version["platform"], R.version["arch"], R.version["os"])
codename <- sub("Codename.\t", "", system2("lsb_release", "-c", stdout = TRUE))
options(HTTPUserAgent = sprintf("R/%s R (%s)", RV, OS))
# register the repositories for The Carpentries and CRAN
options(repos = c(
carpentries = "https://carpentries.r-universe.dev/",
CRAN = paste0("https://packagemanager.posit.co/all/__linux__/", codename, "/latest")
))
})
What if I get errors installing packages?
If you run into errors (non-zero exit status), it probably means that you were missing a C library dependency that needs to be installed via your package manager (i.e. apt). To resolve these issues, scroll back in the log and you might find messages that looks similar to this:
OUTPUT
* installing *source* package ‘xslt’ ...
** using staged installation
Package libexslt was not found in the pkg-config search path.
Perhaps you should add the directory containing `libexslt.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libexslt' found
Using PKG_CFLAGS=-I/usr/include/libxml2
Using PKG_LIBS=-lexslt -lxslt -lxml2
-----------------------------[ ANTICONF ]-------------------------------
Configuration failed to find libexslt library. Try installing:
* deb: libxslt1-dev (Debian, Ubuntu, etc)
* rpm: libxslt-devel (Fedora, CentOS, RHEL)
* csw: libxslt_dev (Solaris)
If libexslt is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libexslt.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
---------------------------[ ERROR MESSAGE ]----------------------------
<stdin>:1:10: fatal error: libxslt/xslt.h: No such file or directory
compilation terminated.
------------------------------------------------------------------------Use the instructions in these logs to install the correct package
from your terminal and then open R or RStudio and retry installing the
packages. For example, for the error above, use
sudo apt install libxslt1-dev.
Test your Workbench installation
To test your installation open RStudio either as the locally installed app or in the browser if using Docker (or launch R if you have not installed RStudio and are not using Docker/devcontainers) and enter the following commands to confirm everything works:
R
rmarkdown::pandoc_version()
tmp <- tempfile()
sandpaper::no_package_cache()
sandpaper::create_lesson(tmp, open = FALSE)
sandpaper::build_lesson(tmp, preview = FALSE, quiet = TRUE)
fs::dir_tree(tmp, recurse = 1)
OUTPUT
[1] '3.10.2'
ℹ Consent for package cache revoked. Use `use_package_cache()` to undo.
→ Creating Lesson in '/tmp/RtmpnRjHyr/file12f34734be05f'...
✔ First episode created in '/tmp/RtmpnRjHyr/file12f34734be05f/episodes/01-introduction.Rmd'
ℹ Workflows up-to-date!
✔ Lesson successfully created in '/tmp/RtmpnRjHyr/file12f34734be05f'
/tmp/RtmpnRjHyr/file12f34734be05f
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE.md
├── README.md
├── config.yaml
├── episodes
│ ├── 01-introduction.Rmd
│ ├── data
│ ├── fig
│ └── files
├── index.md
├── instructors
│ └── instructor-notes.md
├── learners
│ └── setup.md
├── profiles
│ └── learner-profiles.md
└── site
├── DESCRIPTION
├── README.md
├── _pkgdown.yaml
├── built
└── docsIf the installation did not work, please raise an issue on GitHub.
Connect to GitHub
You will need to make sure your git session is connected to GitHub. To do so, you will need to use an SSH or HTTPS protocol. If you already know how to push and pull from GitHub using the command line, you do not need to worry about setting this up.
Docker/devcontainers and git
If you already have your git installation configured outside the Docker/devcontainer environment, the various scripts and commands used in this documentation include mapping your local SSH (and GPG signing) configuration into the container. This means your git commands should “just work” inside the container as they would outside.
If you have any issues, please open an issue on the workbench-docker
GitHub repo and/or ask for help on the workbench channel in
The
Carpentries Slack workspace.
If you do not have this set up, you should choose a protocol and then set them up according to the instructions from GitHub.
It’s recommended to use the SSH protocol, unless you explicitly cannot, e.g. behind an institutional firewall or proxy.
Is GitHub’s Documentation Confusing?
You may find GitHub’s documentation slightly confusing and/or lacking. The following resources are extremely helpful for setting up authentication credentials for your account:
- Remotes in GitHub (Software Carpentry)
- A walkthrough of creating a repository on GitHub and pushing to it via the command line.
- Connect to GitHub (Happy Git With R)
- A walkthrough of creating a throwaway repository that gives you a good idea for the mechanics of working with GitHub.
- Cache credentials for HTTPS (Happy Git With R)
- Clear explanation on how to set up a Personal Access Token and the benefits of using HTTPS. This explains how to do this in both the shell and R.
- Set up keys for SSH (Happy Git With R)
- Clear explanation on what SSH key pairs are and how to set up and connect them with GitHub. This has recommendations using both the shell and RStudio.