=Paper=
{{Paper
|id=Vol-3016/paper11
|storemode=property
|title=Crowd dynamics in internal crowdsourcing: insights from higher education
|pdfUrl=https://ceur-ws.org/Vol-3016/paper11.pdf
|volume=Vol-3016
|authors=Gianluigi Viscusi,Christopher L. Tucci
|dblpUrl=https://dblp.org/rec/conf/stpis/ViscusiT21
}}
==Crowd dynamics in internal crowdsourcing: insights from higher education==
https://ceur-ws.org/Vol-3016/paper11.pdf
Crowd Dynamics in Internal Crowdsourcing: insights from Higher
Education
Gianluigi Viscusia, Christopher L. Tuccia
Imperial College Business School, South Kensington Campus, London, SW7 2AZ, United Kingdom
Abstract
This article aims to study crowd dynamics in small teams. In particular, we are interested in
internal crowdsourcing as a venue for new idea production, highlighting the difference with
regard to other cooperative models based, e.g., on communities. Another goal of the study is
to identify the setting for further experiments in business domains to investigate how crowd
characteristics may lower or increase “crowd capital,” here defined as the total number of
crowd units having a demonstrated effectiveness in idea generation or task achievement. Thus,
the article aims to contribute to the research on coordination in temporary and flash teams. To
this end, the article discusses the main results of six studies carried out with management
students in higher education from 2014 to 2018.
Keywords 1
Collective intelligence, internal crowdsourcing, co-creation, education.
Introduction
This article presents a work in progress that aims to study “crowd” dynamics in small teams. We are
specifically interested in dynamics characterizing internal crowdsourcing [1] as a venue for innovation
and new idea production, questioning the degree of efficiency and effectiveness compared to other
cooperative models based, e.g., on communities [2–4]. Another goal of the study is to provide the setting
for experiments in business domains to investigate how crowd characteristics may lower or increase
“crowd capital,” here defined as the total number of crowd units having a demonstrated effectiveness
in idea generation or task achievement. In particular, the definition focuses on the internal resources of
an organization rather than its inner context as “structure, corporate culture, and political context within
the firm through which ideas for change have to proceed”[5]. The above definition of crowd capital
adopts a more outcome-oriented perspective compared to other definitions emerging from this research
stream [6], complementing the early tentative conceptualization by Prpić, J. & Shukla [7]. Finally, the
article aims to contribute to the research on coordination in temporary groups [8]. The results of the
study, developed through a set of exploratory experiments, aims also to provide insights on the
dynamics emerging from the interactions on internal crowdsourcing platforms that eventually influence
their design as a sociotechnical system [9], thus questioning their impact and the role of users' actions.
The paper is structured as follows. First, the theoretical background of the research is discussed.
Then, the six studies are outlined together with the research method adopted. Finally, a discussion of
the main results and concluding remarks end the paper.
Theoretical Background
The theoretical lens at the basis of the research discussed in this article is the framework and
related typology of “crowd” dynamics proposed by Viscusi and Tucci [10]. The framework considers
the number of participants a sufficient, but not a necessary condition for crowdsourcing, and
Proceedings of the 7th International Workshop on Socio-Technical Perspective in IS Development (STPIS’ 2021), October 14-15, 2021
EMAIL: g.viscusi@imperial.ac.uk (A. 1); c.tucci@imperial.ac.uk (A. 2)
ORCID: 0000-0003-0770-7108 (A. 1); 0000-0001-8733-9530 (A. 2)
© 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
125
distinguishes different types of crowd dynamics according to their growth tendency, degree of seriality,
and the intervening role of properties such as density, equality, and goal orientation for distinguishing
the distribution of agents within and between the different types of “crowds,” namely between groups,
communities, closed crowds, open crowds or multitudes [11]. Closed crowds are usually controlled by
intermediaries, such as, e.g., Innocentive, that restrict growth and provide self-established boundaries;
whereas groups such as crowd crystals may potentially lead to any of the other types of crowd.
Considering now (online) communities, they are a well-known knowledge and innovation
management topic for both practitioners and academics [12]. With a community, the role of identity
and beliefs in the community’s cohesiveness is relevant, rendering them difficult when there are
conflicting and heterogeneous goals. Furthermore, a community is “a group of people who trust each
other. Trust, in turn, is confidence that other people will act, in the future, in ways we think are
right...that they have a generalized disposition to do the right thing” [13]. Accordingly, trust emerges
as a relevant characteristic at the basis of a community. It is worth noting that online communities
smooth these characteristics due to their fluidity, fleeting memberships, dynamic boundaries, and
multiple goals [14]; however, such goals, not necessarily conflicting, could give rise to crowd crystals
or an open crowd.
Let us consider groups as the basic unit from which crowds and communities may arise as also
shown by some of the arguments discussed above. A group can be defined as “an aggregate of
organisms in which the existence of all is utilized for the satisfaction of some needs of each [15] ” or
else as a “self-consciously, mutually acknowledging collective with a self-conscious purpose” [16].
Taking these definitions into account, groups such as crowd crystal are characterized by a lack of scale,
fewer ideas, as well as less input, in addition to a low degree of seriality due to the topical rather than
goal orientation. However, a crowd crystal may grow in an unrestricted fashion, losing the serial nature
of the crowd, becoming no longer anonymous, and finally reaching “community” status. It can also
evolve towards either open or closed crowd types, requiring a stronger, specific goal orientation as well
as information capacity [17] in terms of, e.g., analytics and storage capacity, especially for open crowds,
which can Dynamics
be considered actual multitudes.
Group f
Crowd
Group e Group c
Community
Group b Group a
Group d
Collective Intelligence
Figure 1. Emergent dynamics for collective intelligence, groups, communities, and crowds.
At the state of the art, especially from scholars interested in cognitive economy, the concept of
“multitude” has received different definitions, such as, e.g., “a social body in which singularities are
not required to shed their differences to form a common notion” [18]. Moreover, open crowds when
considered as a type of multitude can consequently be seen as a form of organizing for productive
processes [see also: –11]. This form of organizing combines technological expertise and general social
knowledge in a cognitive cultural economy [see also 19], or collective intelligence, that is considered
as “groups of individuals acting collectively in ways that seem intelligent” [20]. In general, collective
intelligence, groups, online communities, and crowds may represent what Hinings, Gegenhuber, and
Greenwood [21] identify as a type of digital innovation in terms of “a novel digital organizational form
126
is a digitally-enabled arrangement of practices, structures, and values constituting an organization's core
and that is appropriate in a given institutional context.” As shown in Figure 1, these crowd types are
further characterized by emergent dynamics, where groups, communities, and crowds can all be
considered different instances of collective intelligence, with groups as the basic unit for the subsequent
set of crowd dynamics previously discussed. Nowadays, those crowd dynamics are relevant for the
sociotechnical design of open innovation and crowdsourcing platforms where the question is on their
role in the enactment of collaboration or competition amongst the involved users [22–24], requiring
“open, socio-technical systems perspectives” ([22], p. 295). Thus, those crowd dynamics address
exploring the problems raised by social and technical aspects along a continuum and analyzing the
emergent variations of the sociotechnical relationships, e.g., through an experimental approach ([25], p
709), as the one adopted in the research presented below. The studies outlined in the following Section
aim to explore these issues in real settings.
Method
This article presents an exploratory study eventually adopting Internet-mediated research [26] and
qualitative experiments [27–29]. Accordingly, this paper discusses six studies (including the pilot
study) carried out in 2014, 2015, 2016, 2017, 2018 in higher education institutions in Italy and
Switzerland. The pilot study was carried out in the 2014 winter semester at EPFL, Switzerland. The
studies involved students in a Master’s course in Management of Information Systems (University of
Milano-Bicocca, Italy, 2016) Information Technology and e-Business Strategy (EPFL, Switzerland,
2014, 2015), and Information Technology & Digital Strategy (EPFL, Switzerland, 2016, 2017, and
2018). The average number of students for each crowdsourcing exercise was 30 and the goal of the
competition was to identify a business model for a new company, discussing it, and motivating it in a
text of 3000 characters. The groups in Switzerland were made up of people with different backgrounds,
with participants from the management course working with people from Life Sciences, Engineering,
Computer Science, Chemistry, Architecture, etc.; whereas in Italy, the groups included mainly
computer scientists and psychologists. The challenge was divided into three phases:
• Submit: every group advanced its proposal. Each user was randomly assigned a temporary
identity to avoid any biases concerning the identity of participants.
• Vote: each user was asked to vote on the other groups’ proposals. While voting, each user could
save comments on the proposals, which were published only during the Discuss phase.
• Discuss: each user was asked to comment on the other proposals.
It is worth noting that the 2018 challenge was divided into only two phases: Submit ideas and Discuss
& develop, with no voting phase. Moreover, the pilot contest was held with the CROWDICITY idea
management platform (https://crowdicity.com), while the subsequent three crowdsourcing exercises
were carried out via the OXWAY platform (https://oxway.co), a crowdsourcing and collective
intelligence platform from a start-up based in Milan, Italy, and London, UK. The fifth study has been
carried out via the VIIMA platform (https://www.viima.com/), an open innovation platform from Viima
Solutions Oy, a company based in Espoo, Finland. Finally, in 2018, the crowdsourcing exercise was
held on the Moodle eLearning platform adopted for the course by using dedicated fora. The different
technological platforms may eventually have a different impact on the crowd dynamics as discussed
below.
Discussion of the studies
During the pilot study, the 30 students were asked to form six groups of five. Every group had to
elect a captain, i.e., the editor of the group’s proposal. The captain could amend her/his group’s proposal
during the Submit phase and/or during the Discuss phase. Posting, voting, commenting was recognized
as class participation. Participants' names were disguised using anonymous identifiers (IDs). At the end
of the challenge, the top group was given a symbolic prize and the maximum grade for this part of the
127
assessment. During the Voting phase, the individuals were anonymous and ignored the instruction to
only vote for the other groups; thus, they favored their own projects, defending their own proposals and
giving them higher votes than the others. In general, despite the limited number of members and their
mutual acquaintance from the class, the groups acted as crowd crystals oriented to crowd types rather
than as a community with factors such as identity, membership, or friendship seeming less relevant with
regard to the unifying focus on the outcome: having the winning proposal supported individually by the
group members and without reference to real team coordination.
The first study after the pilot was carried out in the 2015 winter semester at EPFL. Specifically, from
November 16th to December 9th, students in the Master’s course in Information Technology and e-
Business Strategy were involved in a crowdsourcing exercise. During the voting phase, the individuals
were anonymous and ignored the instruction to only vote for the other groups. Under the cover of
anonymity, they favored their own projects, defending their own proposals and giving them higher votes
than the others (average of 2.8 for outside evaluations vs. 4.0 for own evaluations). This was mitigated
by the later Discuss phase. Referring to the above-mentioned crowd types, the groups acted as a crowd
crystal rather than a community, despite the limited number of members and their mutual acquaintance
from the class. In addition, the analysis of the comments during the Discuss phase demonstrated also in
this case little relevance of factors such as identity, membership, or friendship; whereas having the
winning proposal emerged as a unifying element within the group (they were focused on the outcome).
Although each group was more or less effective in defending, promoting, and improving the group’s
proposal, there was little evidence of team spirit (they supported the proposal but individually and
without reference to real team coordination). Furthermore, we observed the case of groups partitioned
by activity level and pushed by the activity of two members out of five as in the case of the winner of
the challenge. This may imply that a team could be more effective in finding a solution when the team
is only nominally a team, being instead actually a crowd crystal. One implication of this, on one hand,
may question the performance of community-oriented investments in team-building; on the other hand,
it also opens opportunities for organizations to use prior challenges to look for the best contributors and
then gather them in another (crowd crystal) team, potentially producing effective results on similar
topics and issues as well as new “crowd capital” (the total number of crowd units having a demonstrated
effectiveness in idea generation or task achievement).
The second study was carried out in the 2016 spring semester at the University of Milano-Bicocca.
Specifically, from May 27th to June 19th, students in the Master’s course in Management of Information
Systems participated. The 34 students were asked to form 11 groups of two (three groups), three (four
groups), and four (four groups). Compared to the previous studies, in this study, one group of three
people did not vote for itself, and a group of two did not put forward a proposal at all, thus failing to
compete for the prize. Nonetheless, it is worth noting that this latter group had a member who actively
participated in the discussion phase, thus exhibiting a community-like collaborative orientation rather
than a competitive contest one (age, education, and current employment could be worth investigating,
considering the two members of this group were persons returning to higher education and actually
working full time in companies or public administration). Although the teams in this as well as the pilot
study exhibited on average a similar degree of collaboration, the members of the teams of this study
shown higher individual collaboration compared to the pilot study as well as team coordination (Median
Team Collaboration 0.8 and 0.4 respectively). It is worth noting that in both the pilot study and in this
study, the average “collaborative” team had two members. That point could be a first insight worth
investigating in future work for identifying crowd capital units and the appropriate flash team number
in crowd crystals (e.g., three members for teams in a crowd may have 70% of observed collaboration).
The third study was carried out in the 2016 winter semester at EPFL. Specifically, from November
24 to December 18th, students in the Master’s course in Information Technology and Digital Strategy
th
participated. The 25 students were asked to form eight groups of three (seven groups) and four (1
group). The exercise considered in the third study demonstrated the lowest degree of participation by
each group (overall groups median = 0.2) with two groups relying on the activity of only one member
and two groups of three persons having the activity of the other two members close to zero. It is worth
noting that one of these two groups had one of its members unable to access the discussion and the other
was actually the team winning the prize for the crowdsourcing exercise; thus, the case resembles again
what we observed in the pilot study (having the winning proposal supported individually by the group
128
members without real team coordination). Finally, in this study as well, the average collaborative
members were two (two out of three members for five out of eight groups).
The fourth study was carried out in the 2017 winter semester at EPFL. Specifically, from November
23 to December 15th, students in the Master’s course in IT and Digital Strategy participated. The 26
rd
students were asked to form eight groups of three (six groups) and four (two groups) persons. As
mentioned above, the fourth study was carried out through the basic/free version of VIIMA, an open
innovation platform, which did not allow designing the challenge with strict separation between the
different phases, thus resulting in initiatives more suitable for ideation than crowdsourcing.
Consequently, we adapted the structure of the challenge, having only two main continuous phases: the
Submit Ideas phase, and a unified Vote/Discuss & Develop phase. The figure of the captain was kept
also for this challenge. It is worth mentioning that the openness of the structure of the challenge resulted
in a higher degree of participation by each group (overall groups median = 0.5) than the other three
exercises carried out in Switzerland. It is also worth noting that in this study the average “collaborative”
teams had three members (with the full collaboration of members in three out of six groups of two
members), the groups made up of four members relied on the activity of only one member (the
“captain”), and one group of three had none of the members active in the discussion.
Finally, the fifth study was carried out in the 2018 winter semester at EPFL. Specifically, from
November 14th to December 14th, students in the Master’s course in Information Technology and Digital
Strategy participated. The 28 students were asked to form seven groups of four persons. As already
mentioned, the crowdsourcing exercise was conducted on the Moodle eLearning platform adopted for
the course by using dedicated fora and was made up only of the Submit ideas and Discuss & develop,
with no voting phase. While basically confirming what was observed in the previous studies, the 2018
contest demonstrated the relevance of the factors identified by [30], when looking for example at the
participation in the co-creation aspects of the contests, such as discussion and feedback to the proposals
in the online forum (for example in the 2018 study 75% of women were active participants vs. 55% of
men). Finally, among the three productive members, two people were usually more effective in ideation
activities.
Conclusion
In this paper, we have presented the early results of a study where we have investigated crowd
dynamics in small teams through a set of exploratory experiments. Besides the pilot, regarding the
outcome of the five studies, we saw that three members for teams in a crowd may have 70% of observed
collaboration, with two members usually leading the ideation phase, a result worth investigating in
future work for identifying crowd capital units and the appropriate flash team number in crowd crystals.
Considering now characteristics exhibited by the groups in the studies, and especially to trust as a
property of communities, if we look at opportunistic behaviors, in only a few cases did group members
vote exclusively for their own idea (for example, two out of 14 total votes in the fourth study).
Nevertheless, considering the outcome of the challenges, the studies confirmed what was observed in
the pilot study having the winning proposal supported individually by the group members without real
team coordination. Accordingly, for example, in the fourth study, the winning, as well as the three final
selected ideas, were produced by groups with only one member active on the platform during the
discussion phase, while in the third study two groups were relying on the activity of only one member
and two groups of three persons had the activity of the other two members close to zero. Thus, also in
the fourth case that had in place an open innovation environment, the winning group, as well as the
other two finalists, acted as crowd crystals oriented to crowd types rather than as a community, having
the unifying focus on producing the winning proposal supported individually by the group members
and without exhibited team participation in the discussion. However, these groups showed more
engagement and collaboration in the design phase of the proposal rather than in the social and co-
creation aspects of the challenge. Furthermore, considering the 2018 study with a homogeneous set of
groups of four people, the observed collaboration and online participation to the discussion phase was
~60%, with four teams out of seven with only two active members in the ideation activities. Considering
the different countries and platform settings, it is worth noting that the Italian sample exhibited a level
129
of individual collaboration and team coordination with a crowdsourcing platform that in the case of
Switzerland has been observed in the fourth study where an open innovation platform was adopted
instead.
In conclusion, these observations can be confronted with what has been pointed out by
Majchrzak & Malhotra [31] that “requirements for crowdsourced co-creation are not easily
implementable in a participation architecture because of three tensions they raise” that are “Tension 1:
simultaneously encouraging competition and collaboration;” “Tension 2: idea evolution takes time but
crowd members spend little time;” and “Tension 3: creative abrasion requires familiarity with
collaborators; yet crowds consist of strangers” ([31], p.264). However, the studies have shown that at
least Tension 1 may anyway lead to an outcome that in most cases is also the result of co-creation
(although of groups of three people), especially when the crowd as a collective intelligence is
recognized as made up of multiple collective intelligences represented by groups, explicitly identified
and formed (as in the studies), or potentially emergent and dynamically yet to be identified (hypothesis
for future work). As for Tension 2, it nonetheless depends on the evaluation of the outcome of each
challenge (is the winning idea resulting in a real innovation once implemented by the seeker
organization?) and the measurement of the time spent for each challenge by the winners. In the current
studies, the time spent online in the discussion seemed to be irrelevant to the final outcome, but we do
not have data on the time spent offline for the design phase (apart from the comments on the
engagement). Finally, Tension 3 seems confirmed by the studies where they also have shown how the
crowdsourcing nature of the challenges has generally lowered the effect of the acquaintance amongst
the students deriving from being in the same class, and participation increased when an open innovation
platform was adopted instead of a pure crowdsourcing one.
References
1. O. Zuchowski, O. Posegga, D. Schlagwein, & K. Fischbach, Internal crowdsourcing: conceptual
framework, structured review, and research agenda. Journal of Information Technology, 31 (2016)
166–184. https://doi.org/10.1057/jit.2016.14.
2. E. von Hippel, Innovation Communities. Democratizing Innovation (MIT Press, 2005), pp. 93–
106. https://doi.org/10.1007/978-3-642-22128-6.
3. T. Felin, K. R. Lakhani, & M. L. Tushman, Firms, crowds, and innovation. Strategic Organization,
15 (2017) 119–140. https://doi.org/10.1177/1476127017706610.
4. S. S. Levine & M. J. Prietula, Open Collaboration for Innovation: Principles and Performance.
Organization Science, 25 (2014) 1414–1433.
5. A. M. Pettigrew, Context and action in the transformation of the firm. Journal of Management
Studies, 24 (1987) 649–670.
6. R. Lenart-Gansiniec, Crowd Capital – Conceptualisation Attempt. International Journal of
Contemporary Management; Issue 15(2), (2016).
7. J. Prpic & P. Shukla, The Theory of Crowd Capital. 2013 46th Hawaii International Conference
on System Sciences, (2013) 3505–3514. https://doi.org/10.1109/HICSS.2013.545.
8. M. A. Valentine & A. C. Edmondson, Team Scaffolds: How Mesolevel Structures Enable Role-
Based Coordination in Temporary Groups. Organization Science, 26 (2014) 405–422.
https://doi.org/10.1287/orsc.2014.0947.
9. E. Mumford, The story of socio‐technical design: Reflections on its successes, failures and
potential. Information systems journal, 16 (2006) 317–342.
10. G. Viscusi & C. Tucci, Three’s a Crowd? In C. Tucci, A. Afuah, & G. Viscusi,eds., Creating and
Capturing Value through Crowdsourcing (Oxford University Press, 2018).
11. P. Virno, A Grammar Of The Multitude - For an Analysis of Contemporary Forms of Life
(Cambridge, MA, USA: Semiotext(e) / Foreign Agents - MIT Press, 2004).
12. J. West & K. R. Lakhani, Getting Clear About Communities in Open Innovation. Industry and
Innovation, 15 (2008) 223–231. https://doi.org/10.1080/13662710802033734.
13. C. Heckscher, Trust in a Complex World (Oxford, UK: Oxford University Press, 2015).
130
14. S. Faraj, S. L. Jarvenpaa, & A. Majchrzak, Knowledge collaboration in online communities.
Organization Science, 22 (2011) 1224–1239. https://doi.org/10.1287/orsc.1100.0614.
15. R. B. Cattell, D. R. Saunders, & G. F. Stice, The dimensions of syntality in small groups. Human
Relations, 6 (1953) 331–356.
16. I. M. Young, Gender as Seriality: Thinking about Women as a Social Collective. Signs: Journal of
Women in Culture and Society, 19 (1994) 713–738.
17. C. Batini, M. Castelli, G. Viscusi, C. Cappiello, & C. Francalanci, Digital Information Asset
Evaluation: A Case Study in Manufacturing. SIGMIS Database, 49 (2018) 19–33.
https://doi.org/10.1145/3242734.3242737.
18. N. Tampio, Assemblages and the Multitude: Deleuze, Hardt, Negri, and the Postmodern Left.
European Journal of Political Theory, 8 (2009) 383–400.
https://doi.org/10.1177/1474885109103850.
19. C. Vercellone, From formal subsumption to general intellect: elements for a marxist reading of the
thesis of cognitive capitalism. Historical Materialism, 15 (2007) 13–36.
https://doi.org/10.1163/156920607X171681.
20. T. W. Malone & M. S. Bernstein, Collective Intelligence Handbook (MIT Press Cambridge, MA,
2015).
21. B. Hinings, T. Gegenhuber, & R. Greenwood, Digital innovation and transformation: An
institutional perspective. Information and Organization, 28 (2018) 52–61.
https://doi.org/10.1016/j.infoandorg.2018.02.004.
22. J. Füller, K. Hutter, J. Hautz, & K. Matzler, User Roles and Contributions in Innovation-Contest
Communities. Journal of Management Information Systems, 31 (2014) 273–308.
https://doi.org/10.2753/MIS0742-1222310111.
23. K. Hutter, J. Hautz, J. Füller, J. Mueller, & K. Matzler, Communitition: The Tension between
Competition and Collaboration in Community-Based Design Contests. Creativity and Innovation
Management, 20 (2011) 3–21. https://doi.org/10.1111/j.1467-8691.2011.00589.x.
24. W. Kathan, K. Hutter, J. Füller, & J. Hautz, Reciprocity vs. Free-Riding in Innovation Contest
Communities: Reciprocity vs. Free-Riding in Innovation Contest Communities. Creativity and
Innovation Management, 24 (2015) 537–549. https://doi.org/10.1111/caim.12107.
25. S. Sarker, S. Chatterjee, X. Xiao, & A. Elbanna, The sociotechnical axis of cohesion for the IS
discipline: Its historical legacy and its continued relevance. Mis Quarterly, 43 (2019) 695–720.
26. S. Hesse-Biber & A. J. Griffin, Internet-Mediated Technologies and Mixed Methods Research:
Problems and Prospects. Journal of Mixed Methods Research, 7 (2013) 43–61.
27. G. Kleining, Das qualitative Experiment. In U. Flick, E. v. Kardoff, H. Keupp, L. v. Rosenstiel, &
S. Wolff,eds., Handbuch qualitative Sozialforschung : Grundlagen, Konzepte, Methoden und
Anwendungen (München: Beltz - Psychologie Verl. Union., 1991), pp. 263–266.
28. G. Kleining & H. Witt, The qualitative heuristic approach: A methodology for discovery in
psychology and the social sciences. Rediscovering the method of introspection as an example.
Forum Qualitative Sozialforschung/Forum: Qualitative Social Research (2000).
29. G. Kleining & H. Witt, Discovery as basic methodology of qualitative and quantitative research.
Forum Qualitative Sozialforschung/Forum: Qualitative Social Research (2001).
30. A. Woolley & T. Malone, What makes a team smarter? More women. Harvard Business Review,
(2011).
31. A. Majchrzak & A. Malhotra, Towards an information systems perspective and research agenda on
crowdsourcing for innovation. The Journal of Strategic Information Systems, 22 (2013) 257–268.
https://doi.org/10.1016/j.jsis.2013.07.004.
131