Analysis of Software and Data Carpentry’s Pre- and Post-Workshop Surveys

Why are learners participating in our workshops?

We are interested in knowing why learners attend our workshops. Respondents were asked to check all that apply for several factors provided in the tables below.

85% of Data Carpentry learners attend workshops to learn skills they can apply to their current work.

Software Carpentry’s curriculum teaches basic lab skills for scientific computing. Their workshops include automation with the Unix shell, version control with Git, and programming with R or Python. These tools help learners to increase the efficiency and reproducibility of their computational work.

Compared to Data Carpentry’s learners, Software Carpentry’s tend to have more experience with the tools covered in the workshops, and learners come to learn new and/or additional topics (72.5%). It is also interesting to note that 81.2% of Software Carpentry respondents are first-time attendees.

What is the current level of satisfaction of the data management practices of our learners before attending our workshops?

The majority (72.5%) of Data Carpentry’s respondents are either unsatisfied or feel neutral with their current data management practices. By data management practices, we include behaviors such as keeping your raw data raw, reusing code, and using databases, queries, and scripts to manage large data sets.

How often do our workshop participants program before attending our workshops?

In terms of current programming usage, 36.9% of learners either never use programming, use programming less than once per year, or no more than several times per year. Only 9.4% program on a daily basis. This is no surprise, as Data Carpentry workshops are intended for novices.

Data Carpentry and Software Carpentry workshop participants often find out about our workshops through institutional mailing lists (64.5% and 48.2% respectively). However, “word-of-mouth” recommendations also play a significant role in populating workshops.

In summary, both Data and Software Carpentry workshop respondents attend workshops to learn about or improve upon their current data management and analysis skills.

Comfort of the environment

The Carpentries is committed to making participation in our workshops a harassment-free experience for everyone, regardless of who learners are, where they are from, or what their experience is with the tools we teach. We establish norms for interaction by having, discussing, and enforcing a Code of Conduct so that our workshops provide open and inclusive learning environments. 79% of Data Carpentry respondents either agree or strongly agree that they felt comfortable learning in their workshop environment, and 87.1% of Software Carpentry’s respondents agreed or strongly agreed that the workshop atmosphere was welcoming.

Interaction with Instructors and Helpers

Data Carpentry respondents were asked to rate their level of agreement with several statements regarding their instructors’ knowledge, instructional method, and enthusiasm. Their responses are in the figure below, and axis labels correspond to the statements as follows:

• Instructors Knowledge: The instructors were knowledgeable about the material being taught.
• Instructors Interacting: I felt comfortable interacting with the instructors.
• Instructors Enthusiastic: The instructors were enthusiastic about the workshop.
• Instructors Clear Answers: I was able to get clear answers to my questions from the instructors.

The largest impact we see is that 96.2% of respondents said they felt comfortable interacting with the instructors. We know that our instructors are the reason why our workshops are so well-received. It is also great to see that 94.8% and 96.7% of respondents felt our instructors were knowledgeable about the material being taught, and were enthusiastic about the workshop, respectively. Lastly, 92.9% of respondents felt they were able to get clear answers to their questions from their instructors.

Software Carpentry respondents were asked to rate how they felt instructors and helpers worked as a team, based on the following criteria:

• Considerate: Instructors/Helpers were considerate.
• Enthusiastic: Instructors/Helpers were enthusiastic.
• Clear Answers: Instructors/Helpers gave clear answers to your questions.
• Communicators: Instructors/Helpers were good communicators.

The two neutral centered plots below provide an analysis of respondent’s answers for both instructors and helpers.

From the figures above, we see that Software Carpentry instructors and helpers are considerate, enthusiastic, give clear answers to questions, and are good communicators. As a whole, our instructors work as a team and are successful in creating a warm and welcoming workshop environment.

Respondent Ability to Perform Computing Tasks

Motivation is important, but being confident in your ability to complete specific computing tasks is an equally important goal of Software Carpentry. The grid below shows respondents’ self-reported ability to complete tasks including:

• Pipes: Using pipes to connect shell commands
• Loops: Writing a ‘for loop’ to automate tasks
  
• Git: Initializing a repository with git
• Function: Writing a function
• Import Library: Importing a library or package in R or Python
• Unit Test: Writing a unit test in Python or R
• SQL Query: Writing an SQL query

It also provides their self-reported level of confidence in being able to complete the tasks above after completing the workshop.

These figures tell us that, before the workshop, between 32.2% and 73.4% of the respondents did not feel they could initialize a repository in Git, write a ‘for loop’ to automate tasks, use pipes to connect shell commands, write a SQL query, and/or write a unit test in R or Python. 29% of learners felt their confidence increased greatly with respect to importing a library or package in R or Python. We consider this significant as it is one of the fundamental skills that allows learners to be successful in the other areas mentioned above.

In summary, respondents experienced increased confidence in their ability to perform specific computing tasks and solve problems, or at least search for answers to problems, as a result of participating in Software Carpentry and Data Carpentry workshops.

Countries

The Carpentries is a global community that has recognized the importance of bringing people to data through high-impact trainings. Though the majority of Data Carpentry respondents report attendeding a workshop in the United States of America (45.3%), we also see that learners attend workshops in African (e.g., Ethiopia 3.7%) and European (e.g., Switzerland 0.6%) countries. Note that we haven’t held workshops in Albania or Afghanistan, learners who selected these countries interpreted this question to indicate their country of origin, or made a mistake when selecting their answers.

In Software Carpentry’s pre-workshop survey, respondents are asked whether or not their workshop will take place in the United States. 45.7% of respondents attended a U.S. workshop.

Learners’ discipline

As previously mentioned, Data Carpentry’s curricula is domain-specific to Ecology, Genomics, Geospatial, and the Social Sciences. We see this in the distribution of respondents by discipline. 35.6% are in the Life Science, while 24.6%, 23.4%, and 23.1% are in Agricultural or Environmental Sciences, Bioinformatics/Genomics, and Biomedical/Health Sciences, respectively.

Software Carpentry’s respondent base also has a majority Life Sciences base; however we also see representation from those working in Psychology, High Performance Computing, and Chemistry.

Learners’ career stage

As many of The Carpentries’ workshops are hosted on university or college campuses and other research-based communities, it is no surprise that the majority of respondents are Graduate Students (47.6% - DC, 35.4% - SWC), Research Staff (16.1% - DC,9.6% - SWC), and Postdoctoral Researchers (1.4% - DC, 12.3% - SWC).

Learners’ operating system

In our workshops, we recommend that learners use their own personal laptop computers. It is important for learners to leave the workshop with their own machine set up to do real work. Our instructors teach on three major platforms: Windows, Mac OS X, and UNIX/Linux. We see a very close representation of Windows (53.3%) and Apple/Mac OS (41.3%) users in our Data Carpentry workshops, and even a few UNIX/Linux users (4%).

Gender and Racial/Ethnic Identity

Gender and racial/ethnic identity information is collected for U.S. participants, as we are keen to increase the number of diverse instructors and learners we serve. Understanding our demographic makeup helps us to understand what communities we reach and what programs we should develop.

Currently, both Data (56.6%) Carpentry and Software (48.9%) Carpentry see strong representation from women in the United States. Where we hope to improve is in reaching the non-White audience, as fewer than 50.5% for Data Carpentry, and 45% for SWC of our respondents are from communities historically underrepresented in the science, technology, engineering, and mathematics (STEM) fields.