Data and Flow from Source to Website

Introduction

This is a vignette that is designed for R package developers who are looking to understand how the data flows between the lesson source and the final website. I am assuming that the person reading this has familiarity with R packaging and R environments.

One of the design philosophies of The Workbench is that if a lesson author or contributor would like to add any sort of metadata or modify a setting in any given lesson, they can do so by editing the source of the lesson with no need to modify their workflow.

A note about the design

The design of this is very much PDD (panic-driven design). I was okay with wracking up technical debt because I knew that I could go back and refactor once I got it released. If I had a chance to go back and refactor, I would gladly do so. The creation of muliple storage objects using function factories was what I knew at the time. I now know that it might be better to use global environments instead. The implementation of this was originally done in pull request #248, which was trying to deduplicate code after the release of version 0.1.0, which was a massive push after finally getting the new website layout.

The flow of data I lay out here could all live in the same package-level environment (or even a package-level R6 object) instead of being implemented as function factories, but that’s a refactor for another day and another maintainer.

Two Sources of Metadata

Metadata is different from content because, while it is not directly related to the content, it has extra information that helps the users of the site navigate the content. For example, each episode page has three piece of metadata embedded as a YAML list at the top of the source file that defines the title along with estimated time for teaching and excercises.

title: "Introduction"
teaching: 5
exercises: 5

In terms of the lesson itself, metadata that is related to the whole lesson is stored in config.yaml. This YAML file is designed to be as flat as possible to avoid common problems with writing YAML. Metadata here include things like the title of the Lesson, the source page of the lesson, the time it was created, and the lesson program it belongs to.

title: "An Example Lesson"
carpentry: "incubator"
created: "2023-11-27"
life-cycle: "pre-alpha"

It also defines optional parameters such as the order of the episodes and other content that can be used to customise how the lesson is built.

episodes:
 - introduction.md
 - first-example.md

handout: true

The thing to know about this metdata and these variables is that they all get passed to {varnish} and are used to control how the lesson website is built along with the metadata.

An introduction to `{varnish}`

In order to understand how to pass data from config.yaml to {varnish}, it is important to first understand the paradigm of how {sandpaper} and {varnish} work together to produce a lesson website. Behind the scenes, we build markdown with {knitr}, render the raw HTML with Pandoc and then use {pkgdown} to insert the HTML and metadata into a template (written in the logicless templating language, Mustache) that can be updated and modified independently of {sandpaper}. This template is called {varnish}.

In the paradigm of {pkgdown}, the HTML template is split up into different components that are rendered separately and then combined into a single layout template. For example, here is the template that defines the layout:

<!-- START: inst/pkgdown/templates/layout.html -->
<!-- Generated by pkgdown: do not edit by hand -->
<!doctype html>
<html lang="{{ lang }}">
  <head>
    {{{ head }}}
  </head>
  <body>
    {{{ header }}}
    <div class="container">
      <div class="row">
        {{{ navbar }}}
        {{{ content }}}
      </div><!--/div.row-->
      {{{ footer }}} {{! the end of div class container is in footer }}
  </body>
</html>
<!-- END:   inst/pkgdown/templates/layout.html-->

You can see that it contains {{{ head }}}, {{{ header }}}, {{{ navbar }}}, {{{ content }}} and {{{ footer }}}. These are all the results of the other rendered templates. For example, here’s the head.html template, which defines the content that goes in the HTML <head> tag (defining titles, metadata, stylesheets, and JavaScript):

    <meta charset="utf-8">
    <title>{{#site}}{{&title}}{{/site}}{{#pagetitle}}: {{&pagetitle}}{{/pagetitle}}</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" type="text/css" href="{{#site}}{{root}}{{/site}}assets/styles.css">
    <script src="{{#site}}{{root}}{{/site}}assets/scripts.js" type="text/javascript"></script>
    <!-- mathjax -->
    <script type="text/x-mathjax-config">
    MathJax.Hub.Config({
      config: ["MMLorHTML.js"],
      jax: ["input/TeX","input/MathML","output/HTML-CSS","output/NativeMML", "output/PreviewHTML"],
      extensions: ["tex2jax.js","mml2jax.js","MathMenu.js","MathZoom.js", "fast-preview.js", "AssistiveMML.js", "a11y/accessibility-menu.js"],
      TeX: {
        extensions: ["AMSmath.js","AMSsymbols.js","noErrors.js","noUndefined.js"]
      },
      tex2jax: {
        inlineMath: [['\\(', '\\)']],
        displayMath: [ ['$$','$$'], ['\\[', '\\]'] ],
        processEscapes: true
      }
    });
    </script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script>
    <!-- Responsive Favicon for The Carpentries -->
    <link rel="apple-touch-icon" sizes="180x180" href="{{#site}}{{root}}{{/site}}apple-touch-icon.png">
    <link rel="icon" type="image/png" sizes="32x32" href="{{#site}}{{root}}{{/site}}favicon-32x32.png">
    <link rel="icon" type="image/png" sizes="16x16" href="{{#site}}{{root}}{{/site}}favicon-16x16.png">
    <link rel="manifest" href="{{#site}}{{root}}{{/site}}site.webmanifest">
    <link rel="mask-icon" href="{{#site}}{{root}}{{/site}}safari-pinned-tab.svg" color="#5bbad5">
    <meta name="msapplication-TileColor" content="#da532c">
    <meta name="theme-color" content="#ffffff">

The templates used by {varnish} will take metadata from one of two sources:

a _pkgdown.yaml file that defines global metadata like language (see the pkgdown metadata section for details.
a list of values passed to the pkgdown::render_page() function. These values can be both global and local.

These get inserted into the template via the pkgdown::render_page() function where the contents of the _pkgdown.yaml file are inserted into the data list as $yaml.

The challenge for {sandpaper} is this: when we have a list of per-lesson global variables that need to be allocated when build_lesson() is run and per-file variables that need to be allocated before every call to build_html().

The solution that we’ve come up with is to store these lists in environments that are encapsulated by function factories, which are detailed in the next section.

Storage Function Factories

There are two types of storage function factories in {sandpaper}: Lesson Store (.lesson_store()) and List Store (.list_store()). Both of these return lists of functions that access their calling environments, which are created when the package is loaded:

List Store

The list store acts like a persistent list that has get() set(), clear(), copy() and update() methods.

snd <- asNamespace("sandpaper")
this_list <- snd$.list_store()
names(this_list)
## [1] "get"    "update" "set"    "clear"  "copy"
class(this_list)
## [1] "list-store"

# to set a list, use a NULL key
this_list$set(key = NULL, list(
  A = list(first = 1),
  B = list(
    C = list(
      D = letters[1:4],
      E = sqrt(2),
      F = TRUE
    ),
    G = head(sleep)
  )
))

# get the list
this_list$get()
## $A
## $A$first
## [1] 1
## 
## 
## $B
## $B$C
## $B$C$D
## [1] "a" "b" "c" "d"
## 
## $B$C$E
## [1] 1.414214
## 
## $B$C$F
## [1] TRUE
## 
## 
## $B$G
##   extra group ID
## 1   0.7     1  1
## 2  -1.6     1  2
## 3  -0.2     1  3
## 4  -1.2     1  4
## 5  -0.1     1  5
## 6   3.4     1  6

# copy a list so that you can preserve the original list
that_list <- this_list$copy()

# update list elements by adding to them (note: vectors will be replaced)
that_list$update(list(A = list(platform = "MY COMPUTER")))
that_list$get()
## $A
## $A$first
## [1] 1
## 
## $A$platform
## [1] "MY COMPUTER"
## 
## 
## $B
## $B$C
## $B$C$D
## [1] "a" "b" "c" "d"
## 
## $B$C$E
## [1] 1.414214
## 
## $B$C$F
## [1] TRUE
## 
## 
## $B$G
##   extra group ID
## 1   0.7     1  1
## 2  -1.6     1  2
## 3  -0.2     1  3
## 4  -1.2     1  4
## 5  -0.1     1  5
## 6   3.4     1  6

# set nested list elements
that_list$set(c("A", "platform"), "YOUR COMPUTER")

# note that modified copies do not modify the originals
that_list$get()[["A"]]
## $first
## [1] 1
## 
## $platform
## [1] "YOUR COMPUTER"
this_list$get()[["A"]]
## $first
## [1] 1

Lesson Store

This creates the object containing the pegboard::Lesson object, that we can use to extract episode-specific metadata along with the text of the questions. It is a special object because when a lesson is set with this object, it will additionally set the other global data.

When {sandpaper} is loaded, the List Store and Lesson Store objects are created and live in the {sandpaper} namespace as long as the session is active.

snd <- asNamespace("sandpaper")
some_lesson <- snd$.lesson_store()
names(some_lesson)
## [1] "get"   "valid" "set"   "clear"

Example

All of these metadata are collected and stored in memory before the lesson is built (triggered during validate_lesson()), which are accessible via the objects defined by the internal sandpaper:::set_globals(). Here I will set up an example lesson that I will use to demonstrate these global variables. Please note that I will be using internal functions in this demonstration. These internal functions are not guaranteed to be stable outside of the context of {sandpaper}. Using the asNamespace("sandpaper") function allows me to create a method for accessing the internal functions:

library("sandpaper")
snd <- asNamespace("sandpaper")

# create a new lesson
lsn <- create_lesson(tempfile(), name = "An Example Lesson",
  rstudio = TRUE, open = FALSE, rmd = FALSE)
# add a new episode
create_episode_md(title = "First Example", add = TRUE, path = lsn, open = FALSE)
## /tmp/RtmptMh1k2/file2bf11050a230/episodes/first-example.md

Within {sandpaper}, there are environments that contain metadata related to the whole lesson called .store, .resources, instructor_globals, learner_globals, and this_metadata. Before the lesson is validated, these values are empty:

snd <- asNamespace("sandpaper")
class(snd$.store)
## [1] "list"
print(snd$.store$get())
## NULL
class(snd$this_metadata)
## [1] "list-store"
snd$this_metadata$get()
## list()
class(snd$.resources)
## [1] "list-store"
snd$.resources$get()
## list()
class(snd$instructor_globals)
## [1] "list-store"
snd$instructor_globals$get()
## list()
class(snd$learner_globals)
## [1] "list-store"
snd$learner_globals$get()
## list()

These will contain the following information:

.store a pegboard::Lesson object that contains the XML representation of the parsed Markdown source files
this_metadata the real and computed metadata associated with the lesson along with the JSON-LD template for generating the metadata in the footer of the lesson.
.resources a list of the availabe files used to build the lesson in the order specified by config.yaml
instructor_globals pre-computed global data that includes the sidebar template, the syllabus, the dropdown menus, and information about the packages used to build the lesson
learner_globals same as instructor_globals, but specifically for learner view

When I run validate_lesson() the first time, all the metadata is collected and parsed from the config.yaml, and the individual episodes and cached.

# first run is always longer than the second
system.time(validate_lesson(lsn))
## ── Validating Fenced Divs ──────────────────────────────────────────────
## ── Validating Internal Links and Images ────────────────────────────────
##    user  system elapsed 
##   0.359   0.028   0.387
system.time(validate_lesson(lsn))
## ── Validating Fenced Divs ──────────────────────────────────────────────
## ── Validating Internal Links and Images ────────────────────────────────
##    user  system elapsed 
##   0.111   0.000   0.111

The validate_lesson() call will pull from the cache in .store if it exists and is valid (which means that nothing in git has changed). If it’s not valid or does not exist, then all of the global storage is initialised in this general cascade:

## validate_lesson()
## └─this_lesson()
##   ├─.store$valid()
##   │ ├─gert::git_diff()
##   │ ├─gert::git_status()
##   │ └─gert::git_log()
##   ├─pegboard::Lesson$new()
##   └─.store$set()
##     └─set_globals()
##       ├─initialise_metadata()
##       │ ├─get_config()
##       │ ├─template_metadata()
##       │ └─this_metadata$set()
##       ├─set_language()
##       │ └─add_varnish_translations()
##       │   └─tr_()
##       │     └─these$translations
##       ├─set_resource_list()
##       │ ├─get_resource_list()
##       │ │ └─get_config()
##       │ └─.resources$set()
##       ├─create_sidebar()
##       ├─create_resources_dropdown()
##       ├─learner_globals$set()
##       └─instructor_globals$set()

Now when we these variables are called, you can see the information stored in them.

Lesson Storage

This function stores the pegboard::Lesson object and is responsible for initialising and resetting the variable cache.

snd <- asNamespace("sandpaper")
class(snd$.store)
## [1] "list"
print(snd$.store$get())
## <Lesson>
##   Public:
##     blocks: function (type = NULL, level = 0, path = FALSE) 
##     built: NULL
##     challenges: function (path = FALSE, graph = FALSE, recurse = TRUE) 
##     children: NULL
##     clone: function (deep = FALSE) 
##     episodes: list
##     extra: list
##     files: active binding
##     get: function (element = NULL, collection = "episodes") 
##     handout: function (path = NULL, solution = FALSE) 
##     has_children: active binding
##     initialize: function (path = ".", rmd = FALSE, jekyll = TRUE, ...) 
##     isolate_blocks: function () 
##     load_built: function () 
##     n_problems: active binding
##     overview: FALSE
##     path: /tmp/RtmptMh1k2/file2bf11050a230
##     reset: function () 
##     rmd: FALSE
##     sandpaper: TRUE
##     show_problems: active binding
##     solutions: function (path = FALSE) 
##     summary: function (collection = "episodes") 
##     thin: function (verbose = TRUE) 
##     trace_lineage: function (episode_path) 
##     validate_divs: function () 
##     validate_headings: function (verbose = TRUE) 
##     validate_links: function () 
##   Private:
##     deep_clone: function (name, value)

We can check if the cache needs to be reset by using the $valid() function inside this object, which checks the git log, git status, and git diff as a global check of the lesson contents. When nothing changes, it returns TRUE:

snd$.store$valid(lsn)
## [1] TRUE

However, if we update the lesson contents in some way by setting a config variable, it will be FALSE, indicating that it needs to be reset:

set_config(c(handout = TRUE), path = lsn, write = TRUE, create = TRUE)
## ℹ Writing to /tmp/RtmptMh1k2/file2bf11050a230/config.yaml
## → NA -> handout: true
snd$.store$valid(lsn)
## [1] FALSE
snd$.store$set(lsn)
snd$.store$valid(lsn)
## [1] TRUE

Metadata

The metadata is used to store the content of config.yaml and to provide computed metadata for a lesson that is included in the footer as a JSON-LD object, which is useful for indexing. Note that this metadata must be duplicated for each page to give the correct URL and identifiers.

snd <- asNamespace("sandpaper")
snd$this_metadata$get()
## $carpentry
## [1] "incubator"
## 
## $title
## [1] "An Example Lesson"
## 
## $created
## [1] "2024-04-19"
## 
## $keywords
## [1] "software, data, lesson, The Carpentries"
## 
## $life_cycle
## [1] "pre-alpha"
## 
## $license
## [1] "CC-BY 4.0"
## 
## $source
## [1] "https://github.com/carpentries/file2bf11050a230"
## 
## $branch
## [1] "main"
## 
## $contact
## [1] "team@carpentries.org"
## 
## $episodes
## [1] "introduction.md"  "first-example.md"
## 
## $metadata_template
##  [1] "{{=<% %>=}}"                                                                                                                                       
##  [2] "{"                                                                                                                                                 
##  [3] "  \"@context\": \"https://schema.org\","                                                                                                           
##  [4] "  \"@type\": \"TrainingMaterial\","                                                                                                                
##  [5] "  \"@id\": \"<% url %>\","                                                                                                                         
##  [6] "  \"inLanguage\": \"<% lang %>\","                                                                                                                 
##  [7] "  \"dct:conformsTo\": \"https://bioschemas.org/profiles/TrainingMaterial/1.0-RELEASE\","                                                           
##  [8] "  \"description\": \"<% desc %><% ^desc %>A Carpentries Lesson teaching foundational data and coding skills to researchers worldwide<% /desc %>\","
##  [9] "  \"keywords\": \"<% keywords %><% ^keywords %>software,data,lesson,The Carpentries<% /keywords %>\","                                             
## [10] "  \"name\": \"<% &pagetitle %>\","                                                                                                                 
## [11] "  \"creativeWorkStatus\": \"active\","                                                                                                             
## [12] "  \"url\": \"<% url %>\","                                                                                                                         
## [13] "  \"identifier\": \"<% url %>\","                                                                                                                  
## [14] "  <% #date %>\"dateCreated\": \"<% created %>\","                                                                                                  
## [15] "  \"dateModified\": \"<% modified %>\","                                                                                                           
## [16] "  \"datePublished\": \"<% published %>\"<% /date %>"                                                                                               
## [17] "}"                                                                                                                                                 
## [18] "<%={{ }}=%>"                                                                                                                                       
## 
## $pagetitle
## [1] "An Example Lesson"
## 
## $date
## $date$created
## [1] "2024-04-19"
## 
## $date$modified
## [1] "2024-04-19"
## 
## $date$published
## [1] "2024-04-19"
## 
## 
## $url
## [1] "https://carpentries.github.io/file2bf11050a230/"
## 
## $citation
## [1] "CITATION"
## 
## $overview
## [1] FALSE
## 
## $handout
## [1] TRUE

This metadata is rendered as JSON-LD and passed as a new variable to {varnish} using the internal fill_metadata_template() function:

writeLines(snd$fill_metadata_template(snd$this_metadata))
## {
##   "@context": "https://schema.org",
##   "@type": "TrainingMaterial",
##   "@id": "https://carpentries.github.io/file2bf11050a230/index.html",
##   "inLanguage": "en",
##   "dct:conformsTo": "https://bioschemas.org/profiles/TrainingMaterial/1.0-RELEASE",
##   "description": "A Carpentries Lesson teaching foundational data and coding skills to researchers worldwide",
##   "keywords": "software, data, lesson, The Carpentries",
##   "name": "An Example Lesson",
##   "creativeWorkStatus": "active",
##   "url": "https://carpentries.github.io/file2bf11050a230/index.html",
##   "identifier": "https://carpentries.github.io/file2bf11050a230/index.html",
##   "dateCreated": "2024-04-19",
##   "dateModified": "2024-04-19",
##   "datePublished": "2024-04-19"
## }

Lesson Resources (files)

The next thing that we store globally are the resources we use to build the lesson. This allows us to avoid needing to constantly read the file system:

snd <- asNamespace("sandpaper")
snd$.resources$get()
## $.
##   /tmp/RtmptMh1k2/file2bf11050a230/CODE_OF_CONDUCT.md 
## "/tmp/RtmptMh1k2/file2bf11050a230/CODE_OF_CONDUCT.md" 
##           /tmp/RtmptMh1k2/file2bf11050a230/LICENSE.md 
##         "/tmp/RtmptMh1k2/file2bf11050a230/LICENSE.md" 
##          /tmp/RtmptMh1k2/file2bf11050a230/config.yaml 
##        "/tmp/RtmptMh1k2/file2bf11050a230/config.yaml" 
##             /tmp/RtmptMh1k2/file2bf11050a230/index.md 
##           "/tmp/RtmptMh1k2/file2bf11050a230/index.md" 
##             /tmp/RtmptMh1k2/file2bf11050a230/links.md 
##           "/tmp/RtmptMh1k2/file2bf11050a230/links.md" 
## 
## $episodes
##    /tmp/RtmptMh1k2/file2bf11050a230/episodes/introduction.md 
##  "/tmp/RtmptMh1k2/file2bf11050a230/episodes/introduction.md" 
##   /tmp/RtmptMh1k2/file2bf11050a230/episodes/first-example.md 
## "/tmp/RtmptMh1k2/file2bf11050a230/episodes/first-example.md" 
## 
## $instructors
##   /tmp/RtmptMh1k2/file2bf11050a230/instructors/instructor-notes.md 
## "/tmp/RtmptMh1k2/file2bf11050a230/instructors/instructor-notes.md" 
## 
## $learners
##   /tmp/RtmptMh1k2/file2bf11050a230/learners/reference.md 
## "/tmp/RtmptMh1k2/file2bf11050a230/learners/reference.md" 
##       /tmp/RtmptMh1k2/file2bf11050a230/learners/setup.md 
##     "/tmp/RtmptMh1k2/file2bf11050a230/learners/setup.md" 
## 
## $profiles
##   /tmp/RtmptMh1k2/file2bf11050a230/profiles/learner-profiles.md 
## "/tmp/RtmptMh1k2/file2bf11050a230/profiles/learner-profiles.md" 
## 
## $`renv/profiles/lesson-requirements`
##                                                                                
## "/tmp/RtmptMh1k2/file2bf11050a230/renv/profiles/lesson-requirements/renv.lock"

Global and Local Variables

The rest are global and local variables that are recorded in the instructor_globals and learner_globals. These are copied for each page and updated with local data (e.g. the sidebar needs to include headings for the current page).

snd <- asNamespace("sandpaper")
snd$instructor_globals$get()
## $aio
## [1] TRUE
## 
## $instructor
## [1] TRUE
## 
## $sidebar
## [1] "<div class=\"accordion accordion-flush\" id=\"accordionFlush1\">\n  <div class=\"accordion-item\">\n    <div class=\"accordion-header\" id=\"flush-heading1\">\n        <a href=\"index.html\">Summary and Schedule</a>\n    </div><!--/div.accordion-header-->\n        \n  </div><!--/div.accordion-item-->\n</div><!--/div.accordion-flush-->\n"  
## [2] "<div class=\"accordion accordion-flush\" id=\"accordionFlush2\">\n  <div class=\"accordion-item\">\n    <div class=\"accordion-header\" id=\"flush-heading2\">\n        <a href='introduction.html'>1. introduction</a>\n    </div><!--/div.accordion-header-->\n        \n  </div><!--/div.accordion-item-->\n</div><!--/div.accordion-flush-->\n"  
## [3] "<div class=\"accordion accordion-flush\" id=\"accordionFlush3\">\n  <div class=\"accordion-item\">\n    <div class=\"accordion-header\" id=\"flush-heading3\">\n        <a href='first-example.html'>2. First Example</a>\n    </div><!--/div.accordion-header-->\n        \n  </div><!--/div.accordion-item-->\n</div><!--/div.accordion-flush-->\n"
## 
## $more
## [1] "<hr><li><a class='dropdown-item' href='reference.html'>Reference</a></li>"
## 
## $resources
## [1] "<hr><li><a class='dropdown-item' href='reference.html'>Reference</a></li>"
## 
## $translate
## $translate$SkipToMain
## [1] "Skip to main content"
## 
## $translate$iPreAlpha
## [1] "Pre-Alpha"
## 
## $translate$PreAlphaNote
## [1] "This lesson is in the pre-alpha phase, which means that it is in early development, but has not yet been taught."
## 
## $translate$AlphaNote
## [1] "This lesson is in the alpha phase, which means that it has been taught once and lesson authors are iterating on feedback."
## 
## $translate$iAlpha
## [1] "Alpha"
## 
## $translate$BetaNote
## [1] "This lesson is in the beta phase, which means that it is ready for teaching by instructors outside of the original author team."
## 
## $translate$iBeta
## [1] "Beta"
## 
## $translate$PeerReview
## [1] "This lesson has passed peer review."
## 
## $translate$InstructorView
## [1] "Instructor View"
## 
## $translate$LearnerView
## [1] "Learner View"
## 
## $translate$MainNavigation
## [1] "Main Navigation"
## 
## $translate$ToggleNavigation
## [1] "Toggle Navigation"
## 
## $translate$Menu
## [1] "Menu"
## 
## $translate$SearchButton
## [1] "Search the All In One page"
## 
## $translate$Setup
## [1] "Setup"
## 
## $translate$KeyPoints
## [1] "Key Points"
## 
## $translate$InstructorNotes
## [1] "Instructor Notes"
## 
## $translate$Glossary
## [1] "Glossary"
## 
## $translate$LearnerProfiles
## [1] "Learner Profiles"
## 
## $translate$More
## [1] "More"
## 
## $translate$LessonProgress
## [1] "Lesson Progress"
## 
## $translate$CloseMenu
## [1] "close menu"
## 
## $translate$EPISODES
## [1] "EPISODES"
## 
## $translate$Home
## [1] "Home"
## 
## $translate$HomePageNav
## [1] "Home Page Navigation"
## 
## $translate$RESOURCES
## [1] "RESOURCES"
## 
## $translate$ExtractAllImages
## [1] "Extract All Images"
## 
## $translate$AIO
## [1] "See all in one page"
## 
## $translate$DownloadHandout
## [1] "Download Lesson Handout"
## 
## $translate$ExportSlides
## [1] "Export Chapter Slides"
## 
## $translate$PreviousAndNext
## [1] "Previous and Next Chapter"
## 
## $translate$Previous
## [1] "Previous"
## 
## $translate$EstimatedTime
## [1] "Estimated time: {icons$clock} {minutes} minutes"
## 
## $translate$Next
## [1] "Next"
## 
## $translate$NextChapter
## [1] "Next Chapter"
## 
## $translate$LastUpdate
## [1] "Last updated on {updated}"
## 
## $translate$EditThisPage
## [1] "Edit this page"
## 
## $translate$ExpandAllSolutions
## [1] "Expand All Solutions"
## 
## $translate$SetupInstructions
## [1] "Setup Instructions"
## 
## $translate$DownloadFiles
## [1] "Download files required for the lesson"
## 
## $translate$ActualScheduleNote
## [1] "The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor."
## 
## $translate$BackToTop
## [1] "Back To Top"
## 
## $translate$SpanToTop
## [1] "<(Back)> To Top"
## 
## $translate$ThisLessonCoC
## [1] "This lesson is subject to the <(Code of Conduct)>"
## 
## $translate$CoC
## [1] "Code of Conduct"
## 
## $translate$EditOnGH
## [1] "Edit on GitHub"
## 
## $translate$Contributing
## [1] "Contributing"
## 
## $translate$Source
## [1] "Source"
## 
## $translate$Cite
## [1] "Cite"
## 
## $translate$Contact
## [1] "Contact"
## 
## $translate$About
## [1] "About"
## 
## $translate$MaterialsLicensedUnder
## [1] "Materials licensed under {license} by {authors}"
## 
## $translate$TemplateLicense
## [1] "Template licensed under <(CC-BY 4.0)> by {template_authors}"
## 
## $translate$Carpentries
## [1] "The Carpentries"
## 
## $translate$BuiltWith
## [1] "Built with {sandpaper_link}, {pegboard_link}, and {varnish_link}"
## 
## $translate$ExpandAllSolutions
## [1] "Expand All Solutions"
## 
## $translate$CollapseAllSolutions
## [1] "Collapse All Solutions"
## 
## $translate$Collapse
## [1] "Collapse"
## 
## $translate$Episodes
## [1] "Episodes"
## 
## $translate$GiveFeedback
## [1] "Give Feedback"
## 
## $translate$LearnMore
## [1] "Learn More"
## 
## 
## $sandpaper_version
##  (0.16.4)
## 
## $sandpaper_cfg
## NULL
## 
## $pegboard_version
##  (0.7.5)
## 
## $pegboard_cfg
## [1] "carpentries/pegboard/tree/a32a7836d4455f407c3cafe8ab95edc636e5e919"
## 
## $varnish_version
##  (1.0.2)
## 
## $varnish_cfg
## [1] "carpentries/varnish/tree/9d98f8ca03c0c0ff0d0f20e75e92eae4d08968d8"
## 
## $sandpaper_link
## <a href="https://github.com/carpentries/sandpaper">sandpaper (0.16.4)</a>
## 
## $pegboard_link
## <a href="https://github.com/carpentries/pegboard/tree/a32a7836d4455f407c3cafe8ab95edc636e5e919">pegboard (0.7.5)</a>
## 
## $varnish_link
## <a href="https://github.com/carpentries/varnish/tree/9d98f8ca03c0c0ff0d0f20e75e92eae4d08968d8">varnish (1.0.2)</a>
## 
## $syllabus
##                        episode timings               path percents
## introduction.md   introduction 00h 00m  introduction.html        0
## first-example.md First Example 00h 12m first-example.html       50
##                         Finish 00h 24m                         100
##                                                                      questions
## introduction.md  How do you write a lesson using R Markdown and `{sandpaper}`?
## first-example.md How do you write a lesson using R Markdown and `{sandpaper}`?
##                                                                               
## 
## $overview
## [1] FALSE
snd$learner_globals$get()
## $aio
## [1] TRUE
## 
## $instructor
## [1] FALSE
## 
## $sidebar
## [1] "<div class=\"accordion accordion-flush\" id=\"accordionFlush1\">\n  <div class=\"accordion-item\">\n    <div class=\"accordion-header\" id=\"flush-heading1\">\n        <a href=\"index.html\">Summary and Setup</a>\n    </div><!--/div.accordion-header-->\n        \n  </div><!--/div.accordion-item-->\n</div><!--/div.accordion-flush-->\n"     
## [2] "<div class=\"accordion accordion-flush\" id=\"accordionFlush2\">\n  <div class=\"accordion-item\">\n    <div class=\"accordion-header\" id=\"flush-heading2\">\n        <a href='introduction.html'>1. introduction</a>\n    </div><!--/div.accordion-header-->\n        \n  </div><!--/div.accordion-item-->\n</div><!--/div.accordion-flush-->\n"  
## [3] "<div class=\"accordion accordion-flush\" id=\"accordionFlush3\">\n  <div class=\"accordion-item\">\n    <div class=\"accordion-header\" id=\"flush-heading3\">\n        <a href='first-example.html'>2. First Example</a>\n    </div><!--/div.accordion-header-->\n        \n  </div><!--/div.accordion-item-->\n</div><!--/div.accordion-flush-->\n"
## 
## $more
## [1] "<li><a class='dropdown-item' href='reference.html'>Reference</a></li>"
## 
## $resources
## [1] "<li><a href='reference.html'>Reference</a></li>"
## 
## $translate
## $translate$SkipToMain
## [1] "Skip to main content"
## 
## $translate$iPreAlpha
## [1] "Pre-Alpha"
## 
## $translate$PreAlphaNote
## [1] "This lesson is in the pre-alpha phase, which means that it is in early development, but has not yet been taught."
## 
## $translate$AlphaNote
## [1] "This lesson is in the alpha phase, which means that it has been taught once and lesson authors are iterating on feedback."
## 
## $translate$iAlpha
## [1] "Alpha"
## 
## $translate$BetaNote
## [1] "This lesson is in the beta phase, which means that it is ready for teaching by instructors outside of the original author team."
## 
## $translate$iBeta
## [1] "Beta"
## 
## $translate$PeerReview
## [1] "This lesson has passed peer review."
## 
## $translate$InstructorView
## [1] "Instructor View"
## 
## $translate$LearnerView
## [1] "Learner View"
## 
## $translate$MainNavigation
## [1] "Main Navigation"
## 
## $translate$ToggleNavigation
## [1] "Toggle Navigation"
## 
## $translate$Menu
## [1] "Menu"
## 
## $translate$SearchButton
## [1] "Search the All In One page"
## 
## $translate$Setup
## [1] "Setup"
## 
## $translate$KeyPoints
## [1] "Key Points"
## 
## $translate$InstructorNotes
## [1] "Instructor Notes"
## 
## $translate$Glossary
## [1] "Glossary"
## 
## $translate$LearnerProfiles
## [1] "Learner Profiles"
## 
## $translate$More
## [1] "More"
## 
## $translate$LessonProgress
## [1] "Lesson Progress"
## 
## $translate$CloseMenu
## [1] "close menu"
## 
## $translate$EPISODES
## [1] "EPISODES"
## 
## $translate$Home
## [1] "Home"
## 
## $translate$HomePageNav
## [1] "Home Page Navigation"
## 
## $translate$RESOURCES
## [1] "RESOURCES"
## 
## $translate$ExtractAllImages
## [1] "Extract All Images"
## 
## $translate$AIO
## [1] "See all in one page"
## 
## $translate$DownloadHandout
## [1] "Download Lesson Handout"
## 
## $translate$ExportSlides
## [1] "Export Chapter Slides"
## 
## $translate$PreviousAndNext
## [1] "Previous and Next Chapter"
## 
## $translate$Previous
## [1] "Previous"
## 
## $translate$EstimatedTime
## [1] "Estimated time: {icons$clock} {minutes} minutes"
## 
## $translate$Next
## [1] "Next"
## 
## $translate$NextChapter
## [1] "Next Chapter"
## 
## $translate$LastUpdate
## [1] "Last updated on {updated}"
## 
## $translate$EditThisPage
## [1] "Edit this page"
## 
## $translate$ExpandAllSolutions
## [1] "Expand All Solutions"
## 
## $translate$SetupInstructions
## [1] "Setup Instructions"
## 
## $translate$DownloadFiles
## [1] "Download files required for the lesson"
## 
## $translate$ActualScheduleNote
## [1] "The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor."
## 
## $translate$BackToTop
## [1] "Back To Top"
## 
## $translate$SpanToTop
## [1] "<(Back)> To Top"
## 
## $translate$ThisLessonCoC
## [1] "This lesson is subject to the <(Code of Conduct)>"
## 
## $translate$CoC
## [1] "Code of Conduct"
## 
## $translate$EditOnGH
## [1] "Edit on GitHub"
## 
## $translate$Contributing
## [1] "Contributing"
## 
## $translate$Source
## [1] "Source"
## 
## $translate$Cite
## [1] "Cite"
## 
## $translate$Contact
## [1] "Contact"
## 
## $translate$About
## [1] "About"
## 
## $translate$MaterialsLicensedUnder
## [1] "Materials licensed under {license} by {authors}"
## 
## $translate$TemplateLicense
## [1] "Template licensed under <(CC-BY 4.0)> by {template_authors}"
## 
## $translate$Carpentries
## [1] "The Carpentries"
## 
## $translate$BuiltWith
## [1] "Built with {sandpaper_link}, {pegboard_link}, and {varnish_link}"
## 
## $translate$ExpandAllSolutions
## [1] "Expand All Solutions"
## 
## $translate$CollapseAllSolutions
## [1] "Collapse All Solutions"
## 
## $translate$Collapse
## [1] "Collapse"
## 
## $translate$Episodes
## [1] "Episodes"
## 
## $translate$GiveFeedback
## [1] "Give Feedback"
## 
## $translate$LearnMore
## [1] "Learn More"
## 
## 
## $sandpaper_version
##  (0.16.4)
## 
## $sandpaper_cfg
## NULL
## 
## $pegboard_version
##  (0.7.5)
## 
## $pegboard_cfg
## [1] "carpentries/pegboard/tree/a32a7836d4455f407c3cafe8ab95edc636e5e919"
## 
## $varnish_version
##  (1.0.2)
## 
## $varnish_cfg
## [1] "carpentries/varnish/tree/9d98f8ca03c0c0ff0d0f20e75e92eae4d08968d8"
## 
## $sandpaper_link
## <a href="https://github.com/carpentries/sandpaper">sandpaper (0.16.4)</a>
## 
## $pegboard_link
## <a href="https://github.com/carpentries/pegboard/tree/a32a7836d4455f407c3cafe8ab95edc636e5e919">pegboard (0.7.5)</a>
## 
## $varnish_link
## <a href="https://github.com/carpentries/varnish/tree/9d98f8ca03c0c0ff0d0f20e75e92eae4d08968d8">varnish (1.0.2)</a>
## 
## $overview
## [1] FALSE

Translations

In the majority of the {varnish} templates are keys that need translations that are in the format of {{ translate.ThingInPascalCase }}:

<!-- START: inst/pkgdown/templates/content-syllabus.html -->
<div class="col-xl-12 col-lg-12">
    <main id="main-content" class="main-content">
      <h1>{{& pagetitle }}</h1>
      <div class="container lesson-content">
        <p>
          {{# translate.LastUpdate }} {{& translate.LastUpdate }} |{{/ translate.LastUpdate }}
          <a href="{{#yaml}}{{source}}/edit/{{branch}}/{{/yaml}}{{file_source}}{{^file_source}}index.md{{/file_source}}">
            {{ translate.EditThisPage }} <i aria-hidden="true" data-feather="edit"></i>
          </a>
        </p>

      {{& readme }}

      {{#syllabus}}
      <section id="schedule">
      <table class="table schedule table-striped" role="presentation">
        <tbody>
          {{#setup}}
          <tr>
            <td></td><td><a href="#setup">{{ translate.SetupInstructions }}</a></td><td> {{ translate.DownloadFiles }}</td>
          </tr>
          {{/setup}}
          {{& syllabus }}
      </table>
      <p>
      {{ translate.ActualScheduleNote }}
      </p>
      </section>
      {{/syllabus}}
      {{#setup}}
      <section id="setup">
      {{{setup}}}
      </section>
      {{/setup}}
    </main>
  </div>
<!-- END  : inst/pkgdown/templates/content-syllabus.html -->

These variables are in {sandpaper} and are expected to exist. Because they are expected to exist, theese variables are generated and stored in the internal environment these$translations by the function establish_translation_vars() when the package is loaded. When the lesson is validated, these variables are translated to the correct language with set_language() and placed in the instructor_globals and learner_globals storage. These variables are passed directly to {varnish} templates, which eventually get processed by {whisker}:

snd <- asNamespace("sandpaper")
whisker::whisker.render("Edit this page: {{ translate.EditThisPage }}",
  data = snd$instructor_globals$get()
)
## [1] "Edit this page: Edit this page"

This is key to building lessons in other languages, regardless of your default language. The lesson author sets the lang: config key to the two-letter language code that the lesson is written in. This gets passed to the set_language() function, which modifies the translations inside the global data, but it does not modify the language of the user session:

snd <- asNamespace("sandpaper")
snd$set_config(c(lang = "es"), path = lsn, create = TRUE, write = TRUE)
## ℹ Writing to /tmp/RtmptMh1k2/file2bf11050a230/config.yaml
## → NA -> lang: 'es'
snd$this_lesson(lsn)
whisker::whisker.render("Edit this page: {{ translate.EditThisPage }}",
  data = snd$instructor_globals$get()
)
## [1] "Edit this page: Mejora esta página"
Sys.getenv("LANGUAGE")
## [1] "en"

Switching the language is controlled entirely from within the lesson config:

snd$set_config(c(lang = "en"), path = lsn, create = TRUE, write = TRUE)
## ℹ Writing to /tmp/RtmptMh1k2/file2bf11050a230/config.yaml
## → lang: 'es' -> lang: 'en'
snd$this_lesson(lsn)
whisker::whisker.render("Edit this page: {{ translate.EditThisPage }}",
  data = snd$instructor_globals$get()
)
## [1] "Edit this page: Edit this page"

Translation Variables

There are 61 translations generated by set_language() that correspond to the following variables in {varnish}:

variable	string
`translate.SkipToMain`	`'Skip to main content'`
`translate.iPreAlpha`	`'Pre-Alpha'`
`translate.PreAlphaNote`	`'This lesson is in the pre-alpha phase, which means that it is in early development, but has not yet been taught.'`
`translate.AlphaNote`	`'This lesson is in the alpha phase, which means that it has been taught once and lesson authors are iterating on feedback.'`
`translate.iAlpha`	`'Alpha'`
`translate.BetaNote`	`'This lesson is in the beta phase, which means that it is ready for teaching by instructors outside of the original author team.'`
`translate.iBeta`	`'Beta'`
`translate.PeerReview`	`'This lesson has passed peer review.'`
`translate.InstructorView`	`'Instructor View'`
`translate.LearnerView`	`'Learner View'`
`translate.MainNavigation`	`'Main Navigation'`
`translate.ToggleNavigation`	`'Toggle Navigation'`
`translate.Menu`	`'Menu'`
`translate.SearchButton`	`'Search the All In One page'`
`translate.Setup`	`'Setup'`
`translate.KeyPoints`	`'Key Points'`
`translate.InstructorNotes`	`'Instructor Notes'`
`translate.Glossary`	`'Glossary'`
`translate.LearnerProfiles`	`'Learner Profiles'`
`translate.More`	`'More'`
`translate.LessonProgress`	`'Lesson Progress'`
`translate.CloseMenu`	`'close menu'`
`translate.EPISODES`	`'EPISODES'`
`translate.Home`	`'Home'`
`translate.HomePageNav`	`'Home Page Navigation'`
`translate.RESOURCES`	`'RESOURCES'`
`translate.ExtractAllImages`	`'Extract All Images'`
`translate.AIO`	`'See all in one page'`
`translate.DownloadHandout`	`'Download Lesson Handout'`
`translate.ExportSlides`	`'Export Chapter Slides'`
`translate.PreviousAndNext`	`'Previous and Next Chapter'`
`translate.Previous`	`'Previous'`
`translate.EstimatedTime`	`'Estimated time: {icons$clock} {minutes} minutes'`
`translate.Next`	`'Next'`
`translate.NextChapter`	`'Next Chapter'`
`translate.LastUpdate`	`'Last updated on {updated}'`
`translate.EditThisPage`	`'Edit this page'`
`translate.ExpandAllSolutions`	`'Expand All Solutions'`
`translate.SetupInstructions`	`'Setup Instructions'`
`translate.DownloadFiles`	`'Download files required for the lesson'`
`translate.ActualScheduleNote`	`'The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.'`
`translate.BackToTop`	`'Back To Top'`
`translate.SpanToTop`	`'<(Back)> To Top'`
`translate.ThisLessonCoC`	`'This lesson is subject to the <(Code of Conduct)>'`
`translate.CoC`	`'Code of Conduct'`
`translate.EditOnGH`	`'Edit on GitHub'`
`translate.Contributing`	`'Contributing'`
`translate.Source`	`'Source'`
`translate.Cite`	`'Cite'`
`translate.Contact`	`'Contact'`
`translate.About`	`'About'`
`translate.MaterialsLicensedUnder`	`'Materials licensed under {license} by {authors}'`
`translate.TemplateLicense`	`'Template licensed under <(CC-BY 4.0)> by {template_authors}'`
`translate.Carpentries`	`'The Carpentries'`
`translate.BuiltWith`	`'Built with {sandpaper_link}, {pegboard_link}, and {varnish_link}'`
`translate.ExpandAllSolutions`	`'Expand All Solutions'`
`translate.CollapseAllSolutions`	`'Collapse All Solutions'`
`translate.Collapse`	`'Collapse'`
`translate.Episodes`	`'Episodes'`
`translate.GiveFeedback`	`'Give Feedback'`
`translate.LearnMore`	`'Learn More'`

In addition, there are 26 translations that are inserted before they get to {varnish}:

variable	string
`OUTPUT`	`'OUTPUT'`
`WARNING`	`'WARNING'`
`ERROR`	`'ERROR'`
`Overview`	`'Overview'`
`Questions`	`'Questions'`
`Objectives`	`'Objectives'`
`Callout`	`'Callout'`
`Challenge`	`'Challenge'`
`Prereq`	`'Prerequisite'`
`Checklist`	`'Checklist'`
`Discussion`	`'Discussion'`
`Testimonial`	`'Testimonial'`
`Keypoints`	`'Key Points'`
`Show me the solution`	`'Show me the solution'`
`Give me a hint`	`'Give me a hint'`
`Show details`	`'Show details'`
`Instructor Note`	`'Instructor Note'`
`SummaryAndSetup`	`'Summary and Setup'`
`SummaryAndSchedule`	`'Summary and Schedule'`
`AllInOneView`	`'All in One View'`
`PageNotFound`	`'Page not found'`
`AllImages`	`'All Images'`
`Anchor`	`'anchor'`
`Figure`	`'Figure {element}'`
`ImageOf`	`'Image {i} of {n}: {sQuote(txt)}'`
`Finish`	`'Finish'`

pkgdown metadata

Another source of metadata is that created for the pkgdown in a file called site/_pkgdown.yaml.

# ------------------------------------------------------------------ information
# This file was generated by sandpaper version '0.16.4'
# If you want to make changes, please edit '/tmp/RtmptMh1k2/file2bf11050a230/config.yaml'
# ------------------------------------------------------------------ information

title: An Example Lesson
lang: en
home:
  title: Home
  strip_header: yes
  description: ~
template:
  package: varnish
  params:
    time: 2024-04-19 13:11:20 +0000
    source: https://github.com/carpentries/file2bf11050a230
    branch: main
    contact: team@carpentries.org
    license: CC-BY 4.0
    handout: ~
    cp: no
    lc: no
    dc: no
    swc: no
    carpentry: incubator
    carpentry_name: Carpentries Incubator
    carpentry_icon: incubator
    life_cycle: pre-alpha
    pre_alpha: yes
    alpha: no
    beta: no
    stable: no
    doi: ~
    title: An Example Lesson
    created: '2024-04-19'
    keywords: software, data, lesson, The Carpentries
    episodes: ~
    learners: ~
    instructors: ~
    profiles: ~

This file is what allows {pkgdown} to recognise that a website needs to be built. These variables are compiled into the pkg variable that is passed to all downstream files from build_site(). The important elements we use are

$lang stores the language variable
$src_path stores the source path of the lesson (the site/ directory, or the location of the SANDPAPER_SITE environment variable).
$dst_path stores the destination path to the lesson (site/docs). NOTE: this particular path could be changed in build_site() so that we update the name of the path so that it’s less confusing. The docs folder was a mechanism for pkgdown to locally deploy the site without having to affect the package structure (it was also a mechanism to serve GitHub pages in the past).
$meta a list of items passed on to {varnish}

snd <- asNamespace("sandpaper")
pkg <- pkgdown::as_pkgdown(snd$path_site(lsn))
pkg[c("lang", "src_path", "dst_path", "meta")]
## $lang
## [1] "en"
## 
## $src_path
## /tmp/RtmptMh1k2/file2bf11050a230/site
## 
## $dst_path
## /tmp/RtmptMh1k2/file2bf11050a230/site/docs
## 
## $meta
## $meta$title
## [1] "An Example Lesson"
## 
## $meta$lang
## [1] "en"
## 
## $meta$home
## $meta$home$title
## [1] "Home"
## 
## $meta$home$strip_header
## [1] TRUE
## 
## $meta$home$description
## NULL
## 
## 
## $meta$template
## $meta$template$package
## [1] "varnish"
## 
## $meta$template$params
## $meta$template$params$time
## [1] "2024-04-19 13:11:20 +0000"
## 
## $meta$template$params$source
## [1] "https://github.com/carpentries/file2bf11050a230"
## 
## $meta$template$params$branch
## [1] "main"
## 
## $meta$template$params$contact
## [1] "team@carpentries.org"
## 
## $meta$template$params$license
## [1] "CC-BY 4.0"
## 
## $meta$template$params$handout
## NULL
## 
## $meta$template$params$cp
## [1] FALSE
## 
## $meta$template$params$lc
## [1] FALSE
## 
## $meta$template$params$dc
## [1] FALSE
## 
## $meta$template$params$swc
## [1] FALSE
## 
## $meta$template$params$carpentry
## [1] "incubator"
## 
## $meta$template$params$carpentry_name
## [1] "Carpentries Incubator"
## 
## $meta$template$params$carpentry_icon
## [1] "incubator"
## 
## $meta$template$params$life_cycle
## [1] "pre-alpha"
## 
## $meta$template$params$pre_alpha
## [1] TRUE
## 
## $meta$template$params$alpha
## [1] FALSE
## 
## $meta$template$params$beta
## [1] FALSE
## 
## $meta$template$params$stable
## [1] FALSE
## 
## $meta$template$params$doi
## NULL
## 
## $meta$template$params$title
## [1] "An Example Lesson"
## 
## $meta$template$params$created
## [1] "2024-04-19"
## 
## $meta$template$params$keywords
## [1] "software, data, lesson, The Carpentries"
## 
## $meta$template$params$episodes
## NULL
## 
## $meta$template$params$learners
## NULL
## 
## $meta$template$params$instructors
## NULL
## 
## $meta$template$params$profiles
## NULL

Before being sent to be rendered, it goes through one more transformation, which is where the variable contexts you find in {varnish} come from. The contexts that we use are site for the root path and title, yaml for lesson-wide content, and lang to define the language. Everything else is not used.

dat <- pkgdown::data_template(pkg)
writeLines(yaml::as.yaml(dat[c("lang", "site", "yaml")]))
## lang: en
## site:
##   root: ''
##   title: An Example Lesson
## yaml:
##   time: 2024-04-19 13:11:20 +0000
##   source: https://github.com/carpentries/file2bf11050a230
##   branch: main
##   contact: team@carpentries.org
##   license: CC-BY 4.0
##   handout: ~
##   cp: no
##   lc: no
##   dc: no
##   swc: no
##   carpentry: incubator
##   carpentry_name: Carpentries Incubator
##   carpentry_icon: incubator
##   life_cycle: pre-alpha
##   pre_alpha: yes
##   alpha: no
##   beta: no
##   stable: no
##   doi: ~
##   title: An Example Lesson
##   created: '2024-04-19'
##   keywords: software, data, lesson, The Carpentries
##   episodes: ~
##   learners: ~
##   instructors: ~
##   profiles: ~
##   .present: yes

Zhian N. Kamvar