In the fourth quarter of 2020 The Carpentries collected feedback from community members who took a Carpentries workshop within six months. Find more information about this data collection period on our blog. We are excited to release the results of our long-term survey, and invite community members to use this data to champion The Carpentries far and near.

We released our first long-term survey results in October 2017. You can find the report and its source in the assessment GitHub repository.

We released our latest long-term survey results in April 2020. You can find the report and its source in the assessment GitHub repository.

The results included in this report only cover the data collected in 2020 (between September 28th, 2020 and November 1st, 2020). This report includes the analysis of 263 responses but most questions are optional and specific questions have less answers.

Respondents Career Stage

The majority of long-term survey respondents are graduate students. “Other academic staff” includes Librarians, Research Software Engineers, IT Staff, and Government Research Staff.

Respondents Field of Research, Work, or Study

The largest percentages of long-term survey respondents were from the Life Sciences (19%) followed by Library and Information Science (17%).

Number of Carpentries Workshops Completed

The majority of long-term survey respondents have completed just one Carpentries (Software, Data, or Library) workshop. The percentage of those who have completed only one workshop is up from the previous year.

Time Since Completing a Carpentries Workshop

The majority of long-term survey respondents last completed a Carpentries workshop between 0 and 6 months ago. In 2019, there was a higher percentage of responses from those who had completed a workshop more than a year ago.

Last Carpentries Workshop Attended

Long-term survey respondents were almost evenly split on whether their last Carpentries workshop was a Data or Software Carpentry workshop with Software Carpentry being slightly more common than Data Carpentry. A much smaller percentage of respondents said their last Carpentries workshop was a Library Carpentry workshop.

Content Covered at Last Carpentries Workshop

In 2020, the most common content taught (as reported by long-term survey respondents) was Git followed by the Unix Shell and Python. A much lower percentage of respondents reported their workshop covered R compared with 2019.

Behaviors Adopted

Long-term survey respondents reported that they were most likely to adopt using version control to manage code followed by Other behaviors.

Comparison of Programming Usage Pre- and Post-Carpentries Workshop

Long-term survey respondents reported using programming languages, databases, version control and/or the shell more frequently after taking a Carpentries workshop. Of particular note is the increase in people who now use these tools daily or weekly as well as the drastic decrease in those reporting they don’t use these tools.

Change in Confidence in Tools Covered at Workshop

The vast majority of long-term survey respondents reported that they are more confident using tools covered in the workshop after compared to before.

How Tools Help Respondents

Long-term survey respondents report that using the tools they learned in a Carpentries workshop are improving their overall efficiency as well as their ability to analyze and manage data.

Carpentries Workshop Contributing to Research

For about a quarter of long-term survey respondents, attending a Carpentries workshop has contributed to their writing of a research article, thesis, dissertation, or grant proposal.

Potential Impact on Respondents

As a result of taking a Carpentries workshop, the majority of long-term survey respondents agree that they:

  • have used skills learned at the workshop to advance their career;
  • improved their coding practices;
  • have gained confidence in working with data;
  • made their analyses more reproducible; and
  • been motivated to seek more knowledge about the tools learned.

Involvement in Carpentries Community Post-Workshop

Following the workshop, the majority of long-term survey respondents have subscribed to the Carpentries newsletter. Smaller numbers of respondents have gotten involved in the Carpentries community in other ways, include attending a community discussion, engaging on Twitter, and becoming a workshop helper.

Respondents Participating in Learning Activities

Many long-term survey respondents have continued learning after taking a Carpentries workshop. The most common ways they continued learning were by using both Carpentries and non-Carpentries self-guided lesson material. Smaller numbers participated in additional courses (e.g., online short courses, in-person short courses, or semester-long courses).

Recommendation and Net Promoter Scores

The Net Promoter Score (NPS) for our workshops according to the long-term survey is 47. The NPS varies between -100 and +100. It is calculated by substracting the percentage of respondents who are considered “Promoters” (rating of 9 or 10) and the percentage of respondents who are considered “Detractors” (rating equal or below 6). A positive NPS is deemed good, a NPS above 50 is deemed excellent, and an NPS above 70 is exceptional.

Respondents’ Gender Identity

Note: Gender identity responses apply to U.S. survey respondents only.

A higher percentage of US-based long-term survey respondents are females compared with males or those who are gender variant/non-conforming.

Respondents’ Racial/Ethnic Identity

Note: Racial/ethnic identity responses apply to U.S. survey respondents only.

The highest percentage of US-based long-term survey respondents self-identify as White with smaller percentages identifying as Hispanic or Latino/a, Black or African American, and Asian.

Label on Plot Description in Survey
I prefer not to say I prefer not to say
White White (Having origins in any of the original peoples of Europe, the Middle East, or North Africa.)
Pacific Islander Native Hawaiian or Other Pacific Islander (Having origins in any of the original peoples of Hawaii, Guam, Samoa, or other Pacific Islands.)
Hispanic or Latino/a Hispanic or Latino(a) (A person of Spanish-speaking origin or ancestry and/or Latin American origin or ancestry – includes Portuguese and Brazilians.)
Black or African American Black or African American (Having origins in any of the Black racial groups of Africa – includes Caribbean Islanders and others of African origin.)
Asian Asian (Having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian subcontinent including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, Indonesia, the Philippine Islands, Thailand, and Vietnam.)
American Indian or Alaska Native American Indian or Alaska Native (Having origins in any of the original peoples of North and South America (including Central America), and who maintains a tribal affiliation or community attachment.)

  1. https://orcid.org/0000-0001-5797-5181↩︎

  2. https://orcid.org/0000-0002-9092-966X↩︎

  3. https://orcid.org/0000-0003-4121-2432↩︎