Analysis of Software Carpentry’s Post-Workshop Surveys

Contributors: Kari L. Jordan, Ben Marwick, Naupaka Zimmerman, Erin Becker, and Jonah Duckles
July 2017

For nearly 20 years, Software Carpentry has developed material and trained instructors to teach computing skills to researchers in science, medicine, engineering, and other disciplines. This report is an analysis of the post-workshop survey responses collected for Software Carpentry’s workshops from March 2015 through July 2017. In this two year period, nearly 4,000 responses were collected.

A PDF of the survey questions, the data used in this analysis, and full R code are located on the assessment repo on GitHub. Special thank you to Ben Marwick, Naupaka Zimmerman, Erin Becker, and Jonah Duckles. These individuals made valuable contributions to the code that was used to create the figures in this report.

Community members are invited to contribute code to this analysis. Feel free to use the data and tell us about your findings.

Respondent Demographics

A host of initiatives have been developed and implemented globally to address gender disparities in computing. Software Carpentry’s volunteer instructors have hosted hundreds of workshops since 1998, and the post-workshop survey data shows parity in attendance of males compared to females.

Gender n %
Female 1575 48.5
Male 1597 49.2
Other 10 0.3
Prefer not to say 64 2.0

A breakdown of Software Carpentry’s learners by status is provided below.

42% of Software Carpentry’s post-workshop survey respondents are Graduate Students.

A breakdown of respondents by research domain/field of work or study is provided below. Respondents were asked to check all that apply. The majority of Software Carpentry learners work in Life Sciences.

Research Domain n %
Life Sciences (Genetics, genomics, bioinformatics ) 927 24.9
Life Science - Organismal/systems (ecology, botany, zoology, microbiology, neuroscience) 894 24.0
Planetary sciences (geology, climatology, oceanography, etc.) 245 6.6
Mathematics/statistics 225 6.0
Physics 217 5.8
Civil, mechanical, chemical, or nuclear engineering 167 4.5
Medicine and/or Pharmacy 161 4.3
Chemistry 149 4.0
Social sciences 149 4.0
Library and information science 121 3.2
Economics/business 98 2.6
Humanities 98 2.6
Psychology 88 2.4
Education 79 2.1
High performance computing 79 2.1
Space sciences 33 0.9

Respondent Perception of Workshop Content and Atmosphere

Software Carpentry has developed an interactive instructional approach that includes direct instruction (i.e. explicit teaching and demonstrations), indirect instruction (i.e. problem solving and discovery), and experiential learning. Respondents have mixed feelings about the pace of the workshop they attended, as outlined below.

Pace n %
Just right 1317 38.5
Slightly fast 1057 30.9
Slightly slow 730 21.3
Too fast 157 4.6
Too slow 164 4.8

Respondents were asked to indicate their perception of the balance of lecture to hands-on work in the workshop. A breakdown of their responses is provided below.

Balance: Lecture to Hands-On Work n %
Too much lecture 59 1.7
Slightly too much lecture 354 10.3
Balanced (lecture/hands-on) 2773 81.0
Slightly too much hands-on 204 6.0
Too much hands-on 35 1.0

81% of respondents felt the workshop they attended was well balanced between lecture and hands-on learning.

Learners were asked to rate their level of agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) for the following statements regarding the atmosphere and content of the workshop they attended:

  • Atmosphere: The overall atmosphere of the workshop was welcoming.
  • Material: The material presented matched the workshop description.
  • Recommend: I would recommend this workshop to a friend/colleague.
  • Skills: I learned skills that I will be able to use in my research/work.
  • Worth: The workshop was worth my time.
  • Information: The amount of information covered at the workshop was reasonable for allotted time.

The following Likert chart is an analysis of learner responses to the statements above.

The data strongly suggests that Software Carpentry provides a welcoming environment for its learners where the material not only matches the workshop description, but is worth the time learners spend learning it. Learners acquire skills they are able to apply to their research and/or job function in the time allotted over the two-day period. Lastly, learners feel impressed to recommend the workshop to a friend or colleague.

Respondent Perception of Workshop Instructors and Helpers

A strength of Software Carpentry’s ecosystem is its instructors and helpers. Learners who responded to Software Carpentry’s post-workshop survey were asked to rate how they felt instructors and helpers worked as a team based on the following criteria:

  • Considerate: Instructors/Helpers were considerate.
  • Enthusiastic: Instructors/Helpers were enthusiastic.
  • Communicators: Instructors/Helpers were good communicators.
  • Clear.Answers: Instructors/Helpers gave clear answers to your questions.

The two Likert plots below provide an analysis of respondent answers.