The digitisation of cultural heritage collections has made large parts of our digital heritage available online. However, the collections have often been difficult to access in a meaningful way, still requiring item by item handling of digital images of text to decipher manuscripts and printed materials due to e.g. limited OCR (optical character recognition)/HTR (handwritten text recognition) capabilities or insufficient metadata. While these technologies are under rapid development, in some cases they require training data to learn both machine and handwritten texts, and in some cases it just makes more sense to transcribe texts manually. Enter crowdsourcing: a method where a crowd of people is involved to transcribe, describe or otherwise enrich the digital heritage collections with data [1]. However, the labour and cost efficiency of crowdsourcing in a cultural heritage context has been questioned [2] - is the quality of the crowdsourced results worth the investment in launching and running a crowdsourcing project? The Astrid Lindgren Code project [3] explores Swedish author Astrid Lindgren's original manuscripts in Melin shorthand (stenography). Lindgren's stenography has for long been considered “undecipherable” [4, 5] and has therefore never been subjected to study, making manual interpretation the only existing possibility of accessing the material as well as providing training data for future research [6]. Nevertheless, crowdsourcing has proven to be unexpectedly successful in producing transliterations of Lindgren's stenographed notepads. With 170 volunteers signing up for decoding, prolific attempts during the Spring of 2021 have resulted in a full transliteration of the drafts to novel The Brothers Lionheart (1973) in approximately five weeks. This paper presents the method development securing this successful crowdsourcing process, focusing on the importance of joint ownership, planned communication efforts, and community building through online hackathons. The paper also considers how the particularly challenging circumstances of a pandemic year might have contributed to the avid response from a crowd that normally might have lacked the confidence and time to participate. Transliterating stenography is a particular skill, situated in time and associated with the profession of the former secretary. While substantially limiting the recruitable crowd of volunteers, the paper argues that the requiring of expert skill has been central in the success and the methodological development of the project.
Astrid Lindgren (1907–2002) is a world-famous Swedish author whose writing method was
shorthand. More than 670 shorthand notepads containing all of Lindgren’s literary production are
preserved in the Astrid Lindgren Archives and at The Swedish Institute for Children’s Books. Up
until now, Lindgren's original manuscripts have never been the subject of research, mainly because
“no one has managed to decrypt Lindgren’s stenographic code”, as literary scholar Vivi Edström once
put it [
Although true for most people, the indecipherability of Lindgren’s shorthand does not apply to
everyone. The premise for the ongoing digital humanities project The Astrid Lindgren Code (2020–
2022) [
The mixed methods approach for accessing Lindgren’s shorthand includes HTR experiments,
applied and developed simultaneously with collective transliteration2 through volunteer expert
crowdsourcing [
This paper addresses the process of securing high quality transliteration of Lindgren’s shorthand notepads through citizen science. Our focus is the recruiting, retaining, and utilising of volunteers with expert skills and whose demographics (on the group level) is atypical for crowdsourcing in general.
Mirroring our iterative work process, the paper is structured the following way: First, we provide an overview of the material we are studying and its defining characteristics, as well as the underlying theories and methods we use to approach the subject matter. Second, we describe the four different phases of the project, which each contain methodological considerations, results, and conclusions that led us to the following phase. Lastly, we present our conclusions, reflecting on the initial research questions and main outcomes of the expert crowdsourcing project.
Through the development of the internet, crowdsourcing in a scientific context has become an
increasingly available and popular method for researchers and citizen scientists alike. 4 The method
can “increase the accuracy of computer automated tasks, lower costs, increase the scale of research,
transcend boundaries and borders, produce novel discoveries and increase the speed of research
progression, among other benefits” [
The method is not perfect – common challenges include recruiting and retaining a crowd, finding a diverse enough group to avoid skewed end results, maintaining engagement and interest in the crowdsourcing task over a long time, and completion of a crowdsourced task.5
As with many cases in the information society, technological possibilities can sometimes
overshadow ethical implications. While a research project may be non-commercial, the allocated
1For more about why the manuscripts have not been explored, and Lindgren’s own role in cloaking her shorthand in mystery see M.
Nauwerck, “Storyteller, stenographer, and self-published superstar: how Astrid Lindgren’s multiple roles in book production created the
Lindgren myth”, forthcoming 2022.
2Transliteration is the correct term for the conversion of a text from one script to another which also involves the swapping of letters.
Therefore we use the term transliteration for “transcription” of shorthand.
3Notably, this mixed methods approach is now a built-in feature in the Transkribus platform, and will for example be used in the research
project Gustav’s Hand [
The challenges of the hybrid format of paid and non-paid work are thoroughly discussed by Lund
[
Another aspect of the labour in crowdsourcing is whether it yields enough results to be an
affordable method in humanities research. While the workforce is participating on a volunteer basis,
the infrastructure of crowdsourcing projects (e.g. hosting and supporting a crowdsourcing platform,
adding and editing material to the platform, technical and topical support to volunteers) costs money,
time, and engagement. Experiences from the Transcribe Bentham project (a project to transcribe the
manuscripts of the British moral philosopher Jeremy Bentham) suggest that cultural heritage
crowdsourcing projects need an “ambitious and well thought-through project plan at the very
beginning, and ongoing institutional support, commitment, and resources to successfully meet the
crowdsourcing programme’s goals, or it is unlikely that the cost-avoidance or, indeed, any other aims
will be obtained” [
The cost of crowdsourcing infrastructure can be measured against the results, such as the amount
of transcribed pages or the accuracy rate of transcribed pages or be compared with alternate solutions
to the crowdsourcing task. For instance, the Transcribe Bentham project staff evaluated the economic
investment by comparing crowdsourcing costs with the potential cost of hiring transcribing staff [
A feature of crowdsourcing is that it invites a crowd to participate in a work process, with or
without previous knowledge. Crowdsourcing projects may direct their recruitment efforts so that their
volunteers have a relevant connection to the project (using the Transcribe Bentham project as an
example, the staff reached out to the academic community and to schools [
Expert crowdsourcing (or expert sourcing)8 is, just like its hypernym crowdsourcing, used in many
different ways, spanning from analysing documents written by experts [
For the Astrid Lindgren Code project, a tailored crowd was a necessity due to the specific and nowadays rare knowledge of Melin shorthand. We refer to this method as expert crowdsourcing, meaning that only people with a specific expertise were invited to be part of the crowd in this crowdsourcing effort. 2.2.
Hackathons 6Wikipedia is an openly licensed online encyclopaedia written and edited by volunteers. 7Aside from Transcribe Bentham there are other examples of crowdsourcing of cultural heritage collections which indicate similar conclusions. However, it is difficult to get a detailed overview and Transcribe Bentham serves as a good benchmark for our case due to the strong research connection and the publications related to methodology within the project. 8The two expressions seem to be used interchangeably in various contexts, and there seem to be no consensus on how they differ. 9This example favours “expert sourcing” over “expert crowdsourcing”.
The original meaning of hackathon, a portmanteau of ‘hack’ and ‘marathon’, is a collaborative
event for exploratory programming [
The use of the term was derived from previous experimental crowdsourcing events at Uppsala University Library and was chosen for its connotation to technology to de- dramatise the concept of data and the digital elements of the event, in the hopes of increasing the digital skills and confidence among the participants. As this project progressed we noted that another benefit of using the term hackathon (in lieu of e.g. ‘transcribathon’) is that it is favoured in media storytelling through its contradictory connotations with the project matter (shorthand vs. writing software code) and demography (older generation vs. younger generation).
We chose hackathons as a method because findings from the hackathons at Uppsala University Library suggested that they would increase the sense of ownership among volunteers (which in turn creates more engagement and motivation to continue to contribute) and make the process itself better than if it was pre-defined and static (in terms of workflow efficiency, pedagogical instruction of the workflow, end result of crowdsourcing, or all of these). However, experimental hackathons had not been used to fully transcribe a whole corpus of text. Combining a defined research goal with a crowdsourcing process based on social events and active participation with the process and the task was a method that had not previously been tested.
The Melin system of shorthand was widely used in Sweden during the 20th century. Lindgren
herself learned it as part of her professional secretary training at the Bar-Lock Institute in Stockholm
in 1926–27. Although the skill has become obsolete in Swedish professional life, it is still practised
nationally, for example through Melinska stenografförbundet [
The volunteers were subsequently invited to take part in developing this method from the start, resulting in a more iterative and experimental rather than curated and generalised form of transliteration based on 1) the specificity of Lindgren’s shorthand notepads; and 2) the user experience of the volunteers. In addition to what turned out to be a quick process and a high completion rate, advantages with this approach also include the increased involvement of volunteers in building the project (favouring a sense of joint-ownership) as well as the opportunity for us to explore the potential of letting volunteers play a more dynamic part in the process, as co-developers of methods and tasks.
To a high degree, the volunteers of The Astrid Lindgren Code reflect the medial and historical
context of shorthand in Sweden during the twentieth century. Based on both registration letters and
evaluations, a majority of the volunteers are retired former professionals, born in the 1930s and 40s,
who learned shorthand as part of their professional education, and used it in their careers as
10Since then, the Dickens Code project (2021–2022) [
On a general level, crowdsourcing activities tend to mainly attract male volunteers, most famously
perhaps in the case of Wikipedia [
As previously mentioned, this is reflected in the discourse around The Astrid Lindgren Code
where Swedish media coverage has acknowledged that terms such as “hacking” and “code breaking”
are generally associate with young men, and that an expert crowd of mostly middle aged and older
women in this sense is unexpected [
The undercurrent of women’s coding and grunt work is however an integral part of 20 th century history of computing,12 and today it also serves as a pop culture trope reflected in television shows and films.13 Notably, Lindgren herself worked at the Swedish secret service’s department for letter censorship from 1940 to 1945, an experience that came to influence her literary work and to some extent also her deliberate use of shorthand as a ‘secret language’.
Although the crowdsourcing tasks of The Astrid Lindgren Code are more about interpretation and puzzle solving than decrypting or code breaking in a literal sense, the close relationship between the secretarial profession, problem solving, and technical development has likely been integral to securing participation from an older generation. During the twentieth century, stenography, a technical aid, was eventually replaced by tape recorders, computers and smart devices in the workplace. Volunteers who repeatedly have had to adapt to new technology as part of their profession are arguably more likely to have acquired the necessary digital literacy required for participation.14 3.2.
On a general level, access to the technologies and skills required to participate in online activities
for data creation and sharing has increased during the Covid-19 pandemic. “[T]he crisis has urged
older adults to adopt new technologies to facilitate their tasks, as well as to provide them with an
effective means against loneliness and social isolation caused by the confinement”, as noted by
Martínez-Alcalá [
11An evaluation was sent out on the project’s mailing list in February 2022 and received 35 responses. These responses primarily reflect
experiences from highly motivated individuals who have stayed on as volunteers throughout the project’s progress, making them a relevant
yet inconclusive material for evaluating all participation in the project.
12The first computers were indeed women, and only later was the name designated to machines. Arguably, the pioneer project of digital
humanities was also carried out by female computers. For further reading on the women of Father Roberto Busa's punch card project, see
Terras [
In this section we define and describe the phases of our expert crowdsourcing process. Our aim was to create a social environment around the crowdsourcing task where volunteers could actively contribute to forming the process. We defined four phases of the expert crowdsourcing process: 1) finding a crowd, a platform, and a workflow; 2) introducing and engaging the crowd at hackathons; 3) post-hackathon transliteration and wrap-up; and 4) future development of the project and volunteer initiatives. 4.1.
For the crowdsourcing component of the project, the initial plan was to organise physical hackathons, but since we began in early 2021, we had to adapt to the conditions stipulated by the Covid-19 pandemic. The first phase consisted of finding a crowd, deciding on a crowdsourcing platform and developing workflows for the crowdsourcing process.
Recruitment of volunteers started as the project was presented in an in-depth interview in the
Swedish national radio show Vetenskapsradion forskarliv (P1) [
Recruiting and retaining a crowd is generally considered a challenge for crowdsourcing projects
(for instance, two-thirds of volunteers on the Zooniverse platform make only one classification and do
not return [
Notably, the volunteers often mentioned an identification with or admiration for Astrid Lindgren.
It is noteworthy that this type of emotional reward which indicates an importance of personal
connection to the material is not explicitly mentioned in Estellés-Arolas’ suggestion that recompense
from crowdsourcing “would always look to satisfy one or more of the individual needs mentioned in
Maslow’s pyramid: economic reward, social recognition, self-esteem or to develop individual skills”
[
Our technical specifications for the platform were: preferably open source, easy to manage (i.e. not requiring coding expertise) and easy for volunteers to use. Because of the crowd demographic, we knew that we needed to put special emphasis on the user experience of the platform.
We decided to try out two crowdsourcing platforms which were both feasible options for us as
facilitators: Zooniverse, which is used for crowdsourcing a wide range of subjects and is hosted online
[
As facilitators, Zooniverse would have been advantageous because of the online hosting, the online tutorials for users, and the simple file upload system. Advantages of using Omeka would be the structure for uploading data (the image/item/item set structure is simpler than the manifests used in Zooniverse) and the transparency and availability of data storage and export. 15Although the formation of a core group of more active users is a general tendency in crowdsourcing projects, there are indications that many of the volunteers who originally signed up either found the material too challenging to work with or lost interest for other reasons.
We convened with the expert group of twelve stenographers on two occasions (on 29 January 2021 to test Zooniverse, and on 19 February 2021 to test Omeka). After the first meeting we sent out a survey to gather feedback on the platform, on the project’s structure on the platform, on transliteration instructions, and on the source material itself (the perceived difficulty of reading it, etc.). After the second meeting we sent out another survey with similar questions, as well as questions about how the two platforms compared.
Omeka’s weaknesses were the lack of a Swedish interface and that users had to click a save button to save the transliteration lest all text be lost (this was especially frustrating when one had accidentally navigated away from the Omeka page, e.g. by clicking the back/forward arrows).
In the Zooniverse platform it was difficult to organise the material in a meaningful way as the interface seems to favour a smaller number of files in one item (i.e. a letter with two to four pages is more suitable than a notepad with 60 to 100 pages). The stenographers expressed frustration with lack of context (which is particularly important for shorthand, where interpretation is often based on context) when we uploaded the manuscripts in sets of three pages that appeared to the volunteers in a random order. While we think this issue could have been solved in Zooniverse with a bit more tinkering, it was easier to use Omeka as it already had the hierarchic item organisation that the volunteers requested.
To summarise, both platforms had pros and cons but the deciding factor was the volunteers’ preference for Omeka’s presentation of the manuscripts as well as the comprehensive editing interface, so we used Omeka.
Having decided on a platform, the next step was to ensure that the transliterations were produced
with quality and consistency. Both the stipulations of shorthand transliteration and the future purpose
of the transliterations required particular consideration. Typing up shorthand in a professional context
generally implies both interpretation of intent and the conforming of colloquial and oral elements into
the appropriate written style. Although such a standard would be familiar to the volunteers, it would
eliminate much of what is intriguing about the manuscripts from a literary point of view, such as the
oral aspects of Lindgren’s creative process [
We decided not to use the inbuilt editing features of Omeka, as these would have required the
volunteers to learn and use a dual set of systems for editing. For HTR purposes, we also needed the
transliterations to be very clear on, for example, the exact placement of additions and deletions, and
the inbuilt editing features of Omeka were not sufficient. Furthermore, using the Omeka features
caused confusion when editing the text, as pressing [B] generated the tags “<b>” and “</b>” in the
transliteration, a format that many of the volunteers were not familiar with. For similar reasons we
decided not to use the TEI (the Text Encoding Initiative)16 or similar transcription guidelines as the
learning curve would have been too steep. Instead, we created our own convention for transliteration
and presented it in a manual [
16Text Encoding Initiative, https://tei-c.org/.
With the manuscripts prepared in Omeka, we launched the crowdsourcing platform for the whole group of stenographers. The transliteration activities were centred around a series of hackathons where stenographers could ask questions, collaborate on tricky details in the manuscripts, and continue to develop the workflow. In between the hackathons we communicated through newsletters to encourage continued transliteration and to share news on the project and new guidelines for the workflow.
The inaugural hackathon took place on 7 April 2021, and was followed by five more hackathons on 23 April, 8 May, 20 May, 3 June, and 19 June. The hackathons were two hours long; the length was evaluated at the end of the first hackathon where participants agreed that two hours was an appropriate time and that a longer duration would have been too long to spend in front of a screen. To include as many stenographers as possible, the hackathons were scheduled on different weekdays and hours during the day. Participation was around 20–25 people for each hackathon (except the first one which was attended by around 40 people) and were from a core group of approximately 40 stenographers.
The hackathons were held through the Zoom video online conference tool which was a new kind of software for many participants. Thus, the hackathon participants were confronted with two new pieces of software: Zoom and Omeka. The digital competency skills varied within the group and so a part of each hackathon was dedicated to helping people get online in Zoom, find their bearings with its controls,17 as well as instructing them in the use of Omeka. As the hackathons progressed one-sided support from us was complemented by peer-to-peer support, especially when we started using breakout rooms in Zoom.
Teaching the volunteers how to use the new digital tools was a crucial step in creating an inclusive crowdsourcing environment. On the one hand, they were sought after experts because of their shorthand skills, on the other hand, they were beginners in the digital tools we used. In this environment, self-confidence was mixed (both regarding shorthand skills and digital skills), and it was important to us to emphasise the volunteers’ expertise to motivate the digital tools learning curve. Some volunteers decided to drop out of hackathon participation or out of the project as a whole due to technical difficulties or hardship in understanding the functionalities of Zoom and/or Omeka.
The transliteration workflows during and in between hackathons were continuously developed together with the hackathon participants; the hackathons became a forum for interaction and renegotiation of currently agreed practices. If volunteers had provided input on the process via e-mail, this input was discussed during hackathons to get feedback on the issues. Any new guidelines or recommendations for the project were then communicated through newsletters.
We tried working in different ways during the hackathons to find a suitable form for transliteration workflows, for example solitary work while logged on, working in breakout rooms depending on interest or need (e.g. technical support, coffee break, solving encountered difficulties in the texts), and working in breakout rooms with specific tasks (group work in solving difficult paragraphs and words in an assigned set of notepads).
Some examples of practices that were implemented after the hackathons include: a notepad for difficult passages where volunteers could ask for help and discuss solutions together (this was designed to be a forum for discussion for volunteers in between hackathons, as we decided we could not share personal information such as e-mail addresses with everyone); the manual division of pages to transliterate so that everyone had a more specific task; a review process where a participant could ask for a second reading on specific page(s) they had transliterated; and how to sign and save the transliteration so that it was clear who had transliterated which page. 17E.g. mic on/off, camera on/off, screen sharing, chat function, joining/participating in breakout rooms.
With the hackathons as a backbone for the crowdsourcing effort, we filled the gaps in between with a newsletter to make sure everyone involved got information about changes in the workflow that had been agreed upon on the hackathons. As mentioned in the previous section, each hackathon had 20–25 participants and the full contact list consisted of 170 people.
Drawing on previous experiences from crowdsourcing projects (e.g. [
When working with difficult texts, it may be hard to know when a transliteration (or transcription) is done. Words or sentences may be marked as uncertain and require further review by other stenographers. Deciding on when a transliteration is complete has been an iterative and inconclusive process. The inconclusiveness is partly due to the different uses/meanings of the “final transliteration”: 1) export the text to train an HTR algorithm, 2) create a text suitable for a forthcoming critical edition, and 3) knowing as a volunteer when a page is closed for editing.
Because of these conflicting values we did not end up with a clear definition of when a text is
done. We tried a peer-review system similar to the Library of Congress’ By the People project [
After the last hackathon in June, we created a mailing list using Google Groups to allow for direct connection between volunteers without disclosing their personal data (they had to accept the invite to the group and were made aware of the visibility of their name and email address if they accepted). The mailing list was also created with the hope of ensuring that the crowdsourcing could continue without extensive support from us and instead relying on peer support from the volunteers.
We invited the volunteers to make use of the digital meeting space (accessible through the same link as had been used for all hackathons) to arrange their own hackathons. While no such initiatives were made for the whole group, we know that some of the volunteers met up in smaller groups. These constellations had formed using breakout rooms in the Spring 2021 hackathons.
Most volunteers fell back to solitary work. Some expressed (through emails and the mailing list) that they wanted hackathons to be arranged. We could not meet that need, but the request confirmed conclusions from other crowdsourcing projects: a crowd requires maintenance to be kept together. This became even more important when we were running out of material for the task (something we had not anticipated and therefore not planned for) as the material was an important motivational factor; we know that the lack of material to transliterate caused volunteers to lose interest in the project.
Because of changes in the Covid-19 pandemic restrictions, we were able to organise a physical hackathon in Stockholm as a wrap-up for the crowdsourcing part of the project. The hackathon was arranged as a hybrid event on 8 December 2021, allowing for remote and in-person participation (the number of physical attendees was limited).
About 70 people participated at the event. In addition to the shorthand expert crowd, we also
invited experts from the Astrid Lindgren Society [
The shorthand volunteers expressed joy in participating in the hybrid hackathon, some because of a hackathon taking place after a long hiatus and some emphasised the pleasure of finally meeting us and the other volunteers in person.
For us, it was interesting to arrange a physical event according to what we originally planned in the research application; the Covid-19 pandemic forced us to explore different methods (which in the end may have been more successful for the completion rate) and had the benefits of being able to include volunteers who otherwise would have been restricted from attending due to geographical distances. By using the hybrid format, we could combine the best aspects of in-person and remote participation. 4.4.
The fourth phase is a look into future potential development of this project and the outcome of what we have done so far. How can the data best be utilised for HTR development, and how will we proceed with a genetic/critical edition of The Brothers Lionheart based on the transliterations? Today more than 600 notepads remain in the Astrid Lindgren Archives and there is a keen interest among volunteers to continue transliteration. However, the time frame is an issue: how long will it take to secure funds for continued digitisation and how long will the digitisation process take? Will the interest in the project decrease if the current momentum is lost?
Several volunteers report that primarily, new challenges and tasks are needed to motivate them to continue working with the manuscripts. One solution to all the issues above is to utilise the initiatives created by the volunteers themselves. So far, these have included: lexicons with Lindgren’s shorthand images and typed counterparts in standard Melin from the 1960s; independent research in order to identify unknown texts in the notepads; strategies for recognising when Lindgren has written shorthand in English or German; and developing pedagogical material for teaching shorthand to beginners in collaboration with the Melin shorthand society.
On the one hand, further involvement of the ‘super-users’19 among expert volunteers closes the gap between ‘citizens’ and ‘science’. On the other there are several ethical aspects to consider when lines between volunteer and paid researcher/expert start to blur. If certain individuals are lifted from the crowd of stenographers, what happens to the crowd? Could it potentially have a negative impact on their motivation? We wonder if this is particularly difficult when using expert crowdsourcing, as the crowd of experts is already singled out from the crowd. When recognition and praise is our main compensation to the volunteers, it is a tricky balance to uphold.
An important result of this crowdsourcing project is that it was completed. While there are good
examples of finished crowdsourcing projects (e.g. Anti-Slavery Manuscripts [
The Transcribe Bentham project has made an estimation that the four-year period average of 2,704
transcripts a year would result in a completion of the Bentham texts in 2036 [
Two practical outcomes of the completed transliterations so far are that we have a full text corpus that can be used by the HTR part of the project as well as a full transliteration of shorthand material related to The Brothers Lionheart which can be used for literary analysis. On a broader scale, the transliterated data and its contribution to HTR will contribute to the digital humanities field as a whole, with developed algorithms and training data available as part of larger infrastructures and new projects.
A recommendation we would pass on to other cultural heritage crowdsourcing projects is to centre the crowdsourcing around a limited and well-curated content. This is a bit contradictory to the workflows of mass digitisation of cultural heritage collections where it is sometimes a better strategy to digitise larger quantities at a time,21 but it could be beneficial (for crowdsourcing as well as other public engagement) to break out certain parts of a collection even if the initial result is an “incomplete” digitised collection. Limiting the content and having a clearly defined and attainable end point also limits the social and emotional work that goes down into the upkeep of a crowdsourcing project. 5.2.
Dependence on expert knowledge can actually be an advantage when recruiting, retaining, and motivating a crowd. The targeting of people for their expertise adds value to the user experience and enhances the sense of a specialised and exclusive community, both important factors in motivating the volunteers. From a researcher’s point of view, expert crowdsourcing can be a shortcut to the much sought-after super-users. Within The Astrid Lindgren Code the initiatives of such invested volunteers have provided several opportunities for mixed methods development and sub-projects.
Arguably, the benefits of using experts in crowdsourcing must be balanced against the higher “costs”, i.e. a higher degree of investment and interaction from the project facilitators. In this sense expert labour might ‘cost’ more even when it is free. 5.3.
Except for the hybrid finale, the crowdsourcing process, like many other activities during the Covid-19 pandemic, has been entirely based on remote participation through online software.
On the direct question of how the pandemic has influenced their participation in the project, approximately 50% of the volunteers who responded to our evaluation claim that it did not affect their participation. The other half primarily report that the pandemic has resulted in more time to do the work, because of short term permutations or recommendations to stay at home. One respondent mentioned a general fatigue that drained them of the energy to participate more, whereas others stated that the crowdsourcing activities have felt like something meaningful to do together with others despite isolation, or that participating in the project has been a ray of light in a dreary time.
Although evaluations indicate that the crowdsourcing process would have been successful regardless of the Covid-19 pandemic, it is likely that the pandemic situation favoured broad participation in terms of age and geography, both by contributing to improved digital literacy and by 20Based on the rough estimation that 52 notepads (8% of the shorthand material in the Astrid Lindgren archives) were transliterated in five weeks worth of time. 21To illustrate, in our project the matter of how much and how to digitise the Astrid Lindgren archives was a negotiation with the cultural heritage institution where the material was deposited. creating a need for intellectually challenging and/or socially meaningful activities compatible with remote participation. 5.4.
We can conclude that the hackathons were a successful model to create and maintain a sense of community within the crowd. The hackathons received positive feedback from volunteers both during the spring when they took place and six months after when they were invited to give some general feedback on the project.
The volunteers did not seem to reflect on being co-creators of changes in the workflow, but many of their preferred ways of working are direct results of discussions during and between the hackathons. For instance, many volunteers noted that they preferred the hackathons when they were working in smaller groups to solve problems, which was how the last two hackathons were organised.
While our work focused heavily on including the volunteers in the co-creative process around the crowdsourcing workflow, an unexpected but thrilling outcome was the amount of co-creative projects they themselves initiated (see examples under Phase 4). Regardless of whether these projects are a direct result of our empowerment and co-ownership efforts or of the volunteers’ overall engagement, they have been supported by an accommodating and inclusive atmosphere that encourages initiative and values competence. 5.5. Reaching relationships the unlikely volunteer: communication and personal
We conclude that the citizen science activities have benefitted largely from the media attention given to the Astrid Lindgren Code project. Research communication through press releases and social media as well as continuous interviews have generated public visibility, essential in recruiting volunteers as well as keeping them. Inviting volunteers to participate in interviews and acknowledging their work when results from the project were communicated in social and regular media, have contributed to the overall user-experience and sense of joint ownership.
This is also the case with the continuous communication between volunteers and researchers through email, hackathons, social media, and newsletters. A general focus on reliability, accessibility, inclusion, and appreciation through personal communication has been integral in motivating the volunteers.
The rare demographics of the volunteer group, consisting primarily of Swedish senior women, can possibly shed some new light on how to motivate this target group specifically and why they are often proportionally absent within general crowdsourcing activities although present in other participatory cultures.
A major finding of this study is the importance of a subject or research question which engages the volunteer on a personal level. Here, the professional background and skill set shared with Lindgren as well as the author’s impact on the volunteers through her literary fiction have generated an emotional affinity with the primary material which turned out to be a guarantee for prolific results.
For many volunteers of crowdsourcing the contribution to science is both a community building factor and considered a reward in itself. Therefore, it might be less important whether you apply your knowledge to the classification of ladybirds or counting of hedgehogs. For volunteers of expert crowdsourcing other motivational factors seem to be at play: Does it matter if you transliterate Pippi Longstocking or parliamentary protocols? Does it matter if the task is open for general participation or if participation means being part of an exclusive community? Does it matter if anyone could do the job, or if it is your specific skills that can solve a problem? This study suggests that it does. 6. References