Analysis of Software Carpentry’s Post-Workshop Surveys
Contributors: Kari L. Jordan, Ben Marwick, Naupaka Zimmerman, Erin Becker, and Jonah Duckles
July 2017
For nearly 20 years, Software Carpentry has developed material and trained instructors to teach computing skills to researchers in science, medicine, engineering, and other disciplines. This report is an analysis of the post-workshop survey responses collected for Software Carpentry’s workshops from March 2015 through July 2017. In this two year period, nearly 4,000 responses were collected.
A PDF of the survey questions, the data used in this analysis, and full R code are located on the assessment repo on GitHub. Special thank you to Ben Marwick, Naupaka Zimmerman, Erin Becker, and Jonah Duckles. These individuals made valuable contributions to the code that was used to create the figures in this report.
Community members are invited to contribute code to this analysis. Feel free to use the data and tell us about your findings.
Respondent Demographics
A host of initiatives have been developed and implemented globally to address gender disparities in computing. Software Carpentry’s volunteer instructors have hosted hundreds of workshops since 1998, and the post-workshop survey data shows parity in attendance of males compared to females.
| Gender | n | % |
|---|---|---|
| Female | 1575 | 48.5 |
| Male | 1597 | 49.2 |
| Other | 10 | 0.3 |
| Prefer not to say | 64 | 2.0 |
A breakdown of Software Carpentry’s learners by status is provided below.
42% of Software Carpentry’s post-workshop survey respondents are Graduate Students.
A breakdown of respondents by research domain/field of work or study is provided below. Respondents were asked to check all that apply. The majority of Software Carpentry learners work in Life Sciences.
| Research Domain | n | % |
|---|---|---|
| Life Sciences (Genetics, genomics, bioinformatics ) | 927 | 24.9 |
| Life Science - Organismal/systems (ecology, botany, zoology, microbiology, neuroscience) | 894 | 24.0 |
| Planetary sciences (geology, climatology, oceanography, etc.) | 245 | 6.6 |
| Mathematics/statistics | 225 | 6.0 |
| Physics | 217 | 5.8 |
| Civil, mechanical, chemical, or nuclear engineering | 167 | 4.5 |
| Medicine and/or Pharmacy | 161 | 4.3 |
| Chemistry | 149 | 4.0 |
| Social sciences | 149 | 4.0 |
| Library and information science | 121 | 3.2 |
| Economics/business | 98 | 2.6 |
| Humanities | 98 | 2.6 |
| Psychology | 88 | 2.4 |
| Education | 79 | 2.1 |
| High performance computing | 79 | 2.1 |
| Space sciences | 33 | 0.9 |
Respondent Perception of Workshop Content and Atmosphere
Software Carpentry has developed an interactive instructional approach that includes direct instruction (i.e. explicit teaching and demonstrations), indirect instruction (i.e. problem solving and discovery), and experiential learning. Respondents have mixed feelings about the pace of the workshop they attended, as outlined below.
| Pace | n | % |
|---|---|---|
| Just right | 1317 | 38.5 |
| Slightly fast | 1057 | 30.9 |
| Slightly slow | 730 | 21.3 |
| Too fast | 157 | 4.6 |
| Too slow | 164 | 4.8 |
Respondents were asked to indicate their perception of the balance of lecture to hands-on work in the workshop. A breakdown of their responses is provided below.
| Balance: Lecture to Hands-On Work | n | % |
|---|---|---|
| Too much lecture | 59 | 1.7 |
| Slightly too much lecture | 354 | 10.3 |
| Balanced (lecture/hands-on) | 2773 | 81.0 |
| Slightly too much hands-on | 204 | 6.0 |
| Too much hands-on | 35 | 1.0 |
81% of respondents felt the workshop they attended was well balanced between lecture and hands-on learning.
Learners were asked to rate their level of agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) for the following statements regarding the atmosphere and content of the workshop they attended:
- Atmosphere: The overall atmosphere of the workshop was welcoming.
- Material: The material presented matched the workshop description.
- Recommend: I would recommend this workshop to a friend/colleague.
- Skills: I learned skills that I will be able to use in my research/work.
- Worth: The workshop was worth my time.
- Information: The amount of information covered at the workshop was reasonable for allotted time.
The following Likert chart is an analysis of learner responses to the statements above.
The data strongly suggests that Software Carpentry provides a welcoming environment for its learners where the material not only matches the workshop description, but is worth the time learners spend learning it. Learners acquire skills they are able to apply to their research and/or job function in the time allotted over the two-day period. Lastly, learners feel impressed to recommend the workshop to a friend or colleague.
Respondent Perception of Workshop Instructors and Helpers
A strength of Software Carpentry’s ecosystem is its instructors and helpers. Learners who responded to Software Carpentry’s post-workshop survey were asked to rate how they felt instructors and helpers worked as a team based on the following criteria:
- Considerate: Instructors/Helpers were considerate.
- Enthusiastic: Instructors/Helpers were enthusiastic.
- Communicators: Instructors/Helpers were good communicators.
- Clear.Answers: Instructors/Helpers gave clear answers to your questions.
The two Likert plots below provide an analysis of respondent answers.