-This paper presents results from a survey conducted on a new role in academia: the Research Software Engineer (RSE). The survey provides much needed demographic information about the education, field, gender, job satisfaction and career plans of the people of RSEs. The community is found to be highly educated, derive mainly from the hard sciences, and to be predominantly male. Respondents report satisfaction in their jobs, but indicate that career progression is both difficult and opaque. This paper supports a continued discussion about the experience of RSEs and recommends further investigation into this important community.
The term Research Software Engineer (RSE) was coined
in 2012 in a paper [
broadly, it does not imply that the only people who can inhabit the role derive from a software engineering background. Indeed, most RSEs come from a background other than software engineering.
found that almost 70% of researchers from across domains relied on software for the generation of their results, which shows the fundamental importance of software to research. The availability of this software and its reliability are completely dependent on the skills of the person or people who develop it. Although some academics will choose to adopt the skills they need to engineer reliable software, many will not have either the time or inclination to do so. A more scalable and sustainable approach to embedding these skills in academia is to ensure that academics have access to RSEs.
start an annual survey to collect demographics. This paper shares the results of this survey and uses these results to infer conclusions about the community.
The RSE role is performed by people from a wide range of backgrounds. Anecdotally it appeared that most RSEs came from a research background, albeit one that had allowed them to amass software engineering skills. It appeared that a minority came from a background of formal training in software engineering. This wide range of backgrounds, expertise and education means that it is not currently possible to provide a concise summary of the RSE role which does not overlook some part of the RSE community. Until such a definition exists, it was decided to take an inclusive approach to defining the RSE role by allowing RSEs to self-identify.
the UKRSE Associations mailing list. Two further reminders were sent to the list to elicit responses from the community.
were complete (i.e. where all mandatory questions had been answered). The survey was devised to investigate RSEs in the
from around the world. This first analysis was focuses on
people work, their field and level of education, and their gender.
The most striking result from this survey is that there are far fewer female RSEs than male. Only 11% of the participants were female, and it is assumed that this is representative of the UK community of RSEs. This is a low level of female participation even by the poor standards of UK computer science (14%), which has one of the worst gender disparities in academia.
It is argued that RSEs are more effective at academic software development than a consultant software engineer, because they understand not just software engineering but research too. With 69.5% of participants reporting a doctorate as their highest degree (see Fig. 1), and a further 18.2% reporting a masters degree, it would appear that this argument is sound. The majority of RSEs have worked as a researcher, which will provide them with an intimate knowledge of the research process, its incentives and idiosyncrasies.
mined by asking about the field of their highest degree (see
Fig. 1). The Joint Academic Coding System (JACS) [
The largest proportion of RSEs gained their highest degree in the physical sciences (39.4%, n=130) followed by computer sciences (23.6%, n=78). These subjects alone accounted for 63% of all participants. It is noted that physical sciences and not computer sciences, as was commonly accepted, are the origin of the most participants in the survey, and it would seem reasonable to predict that this reflects the UK RSE community.
ences and computer sciences (11.8%), engineering (7.9%) and biological sciences (7.6%). After this point, no single JACS code accounts for more than 3% of the participants.
7.6% 7.9%
11.8% s e i d u t s d e n i b m o C s e i d u lit s a c o S r e h t O s e c n e i c S l a c i g o il o B g n ir e e n i g n E s e c n e i c S tr e u p m o C d n a l a c it a m e h t a M aspects of both roles in their work, for example best development practices (development side) and paper publications (research side).
the RSE community, it also presented an opportunity to investigate the sustainability of the research software projects on which the RSEs worked. We chose two broad measures to provide an insight into sustainability: the bus factor and technical hand over planning.
who understand a specific software project and could, with only a cursory review of the project, maintain or extend the code. A project with a bus factor of 1 is completely reliant on only one developer. If this developer finds new employment, becomes ill or is hit by the titular bus, then the project will fail.
can be sustained even if a developer leaves.
A technical hand over plan is used to introduce a new developer to a software project. These plans cover basic information, such as the licence and location of the software repository, a description of the software architecture, a summary of development plans and any other information that a new developer would need to understand the software. A project that has written (and maintained) a technical hand over plan can withstand the departure of a developer, even a key developer, significantly better than one without such a plan.
A cause of much controversy in academia is the subject of who should be named on a paper. Researchers are well aware of the importance of papers and citations to their reputation and the progression of their career, and as a consequence they will fiercely argue for inclusion on papers covering work to which they have contributed. One of the complaints heard from the RSE community is that RSEs are not recognised for their, sometimes significant, contribution to research. To investigate the recognition of RSEs in academia the survey focussed on whether they had contributed to research that was published in a paper, and whether they were acknowledged for that contribution.
The results reported for bus factor (see table I) show that the majority of RSEs (46%, n=117) are part of a development team consisting of only one person. It is tempting to seize on this figure and lambast research groups for under-investing in software developers in their team and, in doing so, putting at risk the reliability of their software. We must keep in mind the significant financial restrictions placed on research groups, and that it is unlikely that many researchers would refuse to employ more RSEs if they were given the resources to do so. However, we can conclude that almost half of the research projects included in this survey rely on only one developer, and this is a significant risk to the sustainability of these projects.
group, and just over half of the projects surveyed reported this level of support.
Only 24% of RSEs reported that their software project has a technical hand over plan. Whereas research projects can be forgiven for being over-reliant on too few developers due to the lack of resources, there is little excuse for not developing a technical hand over plan. Almost three quarters of the projects included in this survey report having no technical hand over plan, which puts them at significant risk of sabotaging their own sustainability. The importance of technical hand over plans must be raised with the research community in an attempt to convince more projects to write one.
Although 88.2% (n=284) of RSEs reported contributing to research that was later published in a paper, 30% of these
RSEs more typically receive an acknowledgment in the paper for their contribution (76%, n=278) rather than being named on the paper. It is unusual for an RSE to be named as the lead author on a paper (40% n=147), which could indicate that they are unlikely to conduct their own research. It will be illuminating to conduct follow-up analysis of these RSEs to see where they are publishing their papers and whether the paper focuses on research made possible by software, or on the software itself. More RSEs have been named as a co-author (69% n=253) than as a lead author. 5t0 n e c r e P 25 0
AcknowledgedParticipation Co.author
Lead.author
How to measure the quality of a job has been debated in
psychology for a long time [
we used scales that have been created in [
graduated from Strongly disagree to Strongly agree. Each scale is composed of several so called items (i.e. questions) that each measure one attitude.
There is significant debate about the use of a Likert scale
[
a single concept and to check if all items are effectively
measuring the underlying attitude. The most common used test
for internal consistency is the Cronbach Alpha [
value over the range indicates an overlap of items [
previous research [
We calculated the Cronbach Alpha for each scale we measured and, as shown in table II, found a good internal consistency meaning that it is safe to aggregate the items of each scale into one measure. The aggregation was performed by attributing a score from 1 to 5 for each category and calculating the mean of this aggregate score for the sample (see Fig. 5).
Turnover intention has a low aggregate score (m=0.681, sd=0.825), which indicates that RSEs in the main do not intend to quit their current position. Taken alone, this measure could be understood if RSEs are happy in their current position, if they believe their skills are not needed, or if the market is oversaturated. However, other measures discussed below remove some of these possibilities.
a perceived employability score of m=3.566 (sd=0.922) out of a potential maximum score of 5.0. They also indicate being satisfied that their work is appreciated (item Satisfaction), with a relative high score (m=3.653, sd=0.959), and on the recognition they gain from their colleagues and co-workers (m=3.537, sd=0.988). On the feedback they received from their hierarchy or colleagues (item Feedback), the score is slightly lower (m=3.022, sd=0.831)
It would appear that RSEs are happy in their positions, because they report a low turnover intention, even though their skills are in demand, and satisfaction in the way they are perceived. This runs contrary to the commonly heard complaints of the RSE community, so further analysis must be conducted to understand these results. A series of interviews with RSEs is planned to investigate this issue in more depth.
to promotion within the group. We asked 5 questions to investigate how RSEs viewed their career plan (see the table
Turnover Intention Perceived Employability Satisfaction Recognition Feedback
step in their career plan (item Career.Plan) with a high score in general (m=3.453, sd=1.183) and they also report that their next position is likely to also be an RSE role (item Next.Position), (m=3.285, sd=1.135). This indicates that RSEs are choosing their role and the result for Next.Position could be argued to show that RSEs are choosing, not just a job, but a career in Research Software Engineering. However, they are less positive about the opportunities for promotion (item Promotion) (m=2.315, sd=1.202). They also report a lack of information on the promotion process (item Information) (m=2.353, sd=1.247), which is understandable because the
why they are asked to self-identify, is discussed above. The UKRSE Association1 were the first to encounter this problem and as a consequence had developed a number of questions to help potential RSEs align themselves with the role. These questions are as follows: 1) Are you employed primarily to develop software for research? 2) Do you spend more time developing software than conducting research? 3) Are you employed as a postdoctoral researcher? 4) Are you the person who does computers in your research group?
worked in a research environment but wrote software rather than papers, and were based on the experiences of a group of people who had worked in the RSE role. They were designed to identify aspects that, anecdotally at least, appeared to unite RSEs from all backgrounds. For example, many appeared to have been recruited into postdoctoral positions, and many were referred to as the person ”who does computers” by their less technologically adept colleagues. The above questions are broad, and even colloquial at times, to appeal to the intended audience and present the RSE community as inclusive. RSEs were said to have self-identified once they had joined the
To investigate the effectiveness of the above questions in identifying RSEs, they were included in the RSE survey. The results (see table IX) indicate that, unsurprisingly, the most effective way of identifying RSEs is to focus on software development: 72% (n=241) reported that they spend more time developing software than conducting research, and 58% (n=193) reported that they are employed primarily to develop software for research. The question about the person who does computers was agreed with by 40% (n=136) of participants, which indicates that this colloquial approach appeals to a substantial number of RSEs. Only 28% (n=95) of participants reported that they are employed in a postdoctoral position. It would appear that the widely held belief that most RSEs are employed as postdoctoral researchers is untrue.
would appear that there is significant agreement only with the questions related to time spent performing software development. This result will be communicated with the UKRSE
support of hard evidence. It must first be shown to have gained traction with its target community before that community will have sufficient incentive to contribute to evidence gathering.
it had gained significant interest, membership had grown to a significant size, but the tenets on which the community was based were supported mainly by anecdotal evidence. The RSE survey described in this paper provides evidence for the first time about this important community.
It would appear that many assumptions about the community are correct. They are have an intimate knowledge of research, with 69.5% holding a doctorate, and they mainly derive from a background in the physical science and computer sciences. Anyone who spends time with this community will realise that it is predominantly male, as shown by the small number of female participants to the survey: a mere 11%. This gender imbalance is an issue of much debate across STEM subjects, which provides a potential area of collaboration for the RSE community and route to addressing the issue. It is noted that the RSE community has accepted this issue and is already working to improve representation across its membership.
of the software on which RSEs work. It was found that 46% of RSEs work on projects with a bus factor of 1 and, potentially more worrying, 6% of RSEs work on projects where no technical hand over plan exists. The concepts of software sustainability are still relatively new to many in the research community, and it would appear that there is much work to be done to ensure that the community can sustain its own work - at least according to these broad measures.
in papers. This argument is supported by the survey which found that despite 88.2% of RSEs contributing to a result that appeared in publication, 24% of these RSEs were not even acknowledged in the paper. Paper publications might not be the most appropriate mechanism for acknowledging the work of
to ensure that they are acknowledged for their contribution by conventional measures.
We investigated attitudes towards the RSE career using robust and previously assessed scales. Turnover intention is a critical aspect, because it informs us of fragility of the community. The participants did not express any significant desire to leave their current position, which indicates that the community is robust The affective side (job satisfaction), the cognitive aspect (perceived employability), and the social aspect (recognition) of the RSE role were investigated are were positive.
to stay in the role, they are less positive about the likelihood of promotion and access to information about it. This could be a warning sign of a dead-end career path.
explained by the novelty of their role and the interest this generates. Longitudinal analysis is needed to develop a deeper understanding of this result, but the conclusion based on this survey is that RSEs feel positive about their role.
other than the UK. It would be enlightening to conduct the same analysis of software engineers working in industrial research (e.g. in R&D departments) to a relative measure against which research software engineering can be related.