1 Introduction
In the social sciences, application of set theory has seen a dramatic increase over the
last decade. This can be attributed to the method called “Qualitative Comparative
Analysis (QCA)” [
1
] developed by Charles Ragin [
2, 3
], a political scientist. QCA as
a set-theoretical method models causal relations as subset or superset relations;
necessity and sufficiency and focusses on arriving at causally complex patterns in terms
of equifinality, conjunctural causation and asymmetry [
3-5
]. Initially applied by a
small academic community of sociologists and political scientists, this method has
been widely adopted in the fields of management sciences (e.g. strategic management
[
5
], marketing [
6
]), engineering (e.g. disaster management [
7
]) and recently in the
domain of information systems (e.g. user resistance to IT [
8
], IT business value
research [
9
], digital eco dynamics [
10
] and IT project management [
11
]). Although
developed initially by Ragin [
2
] for qualitative case study researchers (medium
sample size or N < 90), the proponents and supporters of QCA have argued about its
unique advantages over regression-based approaches [
4, 12, 13
] and its application for
analysis of large-N datasets [
12
]. In the adoption trajectory of set theoretical methods
in social sciences [
1
], three variants of QCA methodology (crisp-set QCA(CsQCA),
fuzzy-set QCA (fsQCA) [
3
] and multi-value QCA (MvQCA) [
4
]), and a novel
approach to identifying necessary conditions i.e. NCA [
14
] has surfaced with a number
of software tools helping researchers to conduct set-theoretic social science research
(e.g. R packages like QCA and QCAPro, fs/QCA, Tosmana). A detailed review of
available set-theoretical analysis software can be accessed at
http://www.compasss.org/software.htm. Furthermore, other related domains (e.g.
computer science, forecasting) has also seen a steady rise in the application of fuzzy
set or multi valued logic ever since the concept was initiated by initiated by Lotfi A.
Zadeh [
15
] in 1965 [
16
]. Inspired by this application of set theory across domains, a
number of scholars, e.g. Smithson and Verkuilen [
17
], Vatrapu et.al [
18
] to name a
few, have highlighted key advantages of applying classical set theory [
19
] in general
and fuzzy set theory [
20
] in particular with them being:
(1) Set-theoretical ontology (e.g. Fuzzy Sets) is well suited to conceptualize
vagueness, which is a central aspect of social science constructs. For example, in the
social science domain of marketing, concepts such as brand loyalty, brand
sentiment are vague.
(2) Set-theoretical epistemology is well suited for analysis of social science
constructs that are both categorical and dimensional. That is, set-theoretical approach
is well suited for dealing with different and degrees of a particular type on
construct. For example, social science constructs such as culture, personality, and
emotion are all both categorical and dimensional.
(3) Set-theoretical methodology can help analyze multivariate associations beyond
the conditional means and the general linear model. In addition, set theoretical
approaches analyze human associations prior to relations and this allows for both
quantitative variable centered analytical methods as well as qualitative case study
methods.
(4) Set-theoretical analysis has high theoretical fidelity with most social science
theories, which are usually expressed logically in set-terms. For example, theories
on market segmentation and political preferences are logically articulated as
categorical inclusions and exclusions that natively lend themselves into set theoretical
formalization and analytics.
(5) Set-theoretical approach systematically combines set-wise logical formulation of
social science theories and empirical analysis using statistical models for
continuous variables. For example, in the case of predictive analytics, it is possible to
employ set and fuzzy theory to dynamically construct data points for independent
variables such as brand sentiment (polarity, subjectivity, etc.).
Based on the above developments in set theoretical thinking in social sciences, recent
developments in set visualization techniques (e.g. Upset [
21
], Pathfinder [
22
], Circles
[
23
], sets as configurations [
24
]) and employing methods like Qualitative
Comparative Analysis (QCA), Necessary Condition Analysis (NCA) and Upset, in this
position paper, we propose a new approach to maturity models in the domain of
Information Systems. The rest of this position paper is structured as follows. First, we
discuss QCA as a set-theoretic method (section 2), and introduce a complimentary
method called Necessary Condition Analysis (NCA). Second, we briefly describe the
concept of maturity models, and conceptualize maturity models in set-theoretical
terms of necessary and sufficient conditions. Third, we present our empirical dataset;
showcase the application of Upset in identifying suitable data-set for analysis. Fourth,
we present our proposed approach (steps), discuss our results and analysis. Fifth and
last is conclusion and future research agenda.
Qualitative Comparative Analysis (QCA) and other set-theoretic methods investigate
social phenomena of interest by using sets and the search for set relations[
4
]. Many
researchers ([
13
], [
3
], [
24
], [
25
], [
4
]) advocate for the ontology of a social
phenomenon being framed in terms of set relations, and using set-theoretic methods to
investigate these statements [
4
]. This section briefly presents the basic principles of
set theory (e.g. necessary, sufficient conditions and configurations) and discusses the
two methods applied till date in the social sciences (QCA and NCA).
Firstly, in any set theoretic method, it is very important to identify “necessary
conditions”, as without them the outcomes cannot occur, and other conditions cannot
compensate for their absence [
14
]. A necessary condition is an antecedent condition that is
a superset of the outcome [
3
]. As shown in Table1, depending on the set formulation
(i.e. crisp or fuzzy), in a perfect world one could detect a necessary condition, just by
looking at the graph. With both crisp and fuzzy sets (Table 1: 1a & 1c), the necessary
condition is represented as a superset relation and indicated as Xi ≥ Yi (X is a superset
of Y). Another way of identifying necessary conditions is by visualizing crisp sets in a
tabular format (1d). A test for necessity essentially requires us to look at only the first
row (cells 1 & 2), while cells 3 and 4 are completely irrelevant as shown in 1d. Test
for necessity is followed by a test for sufficiency (1b, 1e, and 1f). Test for sufficiency
however proceeds from the observation of some condition(s) X to the observation of
the outcome Y [
1
] as illustrated in Table 1.
Once the necessary and sufficient conditions are identified, set-theoretical social
science researchers focus on configurations of how relevant these conditions fit together
to achieve the desired outcome (Y). In the real world, empirical data about a certain
social phenomenon is often noisy, and in order to detect necessary and sufficient
conditions, QCA researchers have developed measures of consistency, coverage [
3
],
relevance, trivialness [
25
] and also some diagnostics to detect paradoxical
relations[
1
]. QCA adopts Boolean minimization using the Quine-McCluskey algorithm
combined with qualitative counterfactual analysis to arrive at the final solution [
1, 3,
4
]. This final solution is presented as the optimal configuration for achieving the
desired outcome (Y). However, the ultimate goal of QCA is to analyze set-theoretic
sufficiency relations [
2
] and researchers applying QCA are sometimes accused of
ignoring necessary but not sufficient conditions. Moreover, calibration of the original
data into set-memberships and the construction of the truth table forms central core of
this method. Since calibration involves transforming the original dataset, some
scholars(e.g. [
26
], [
25
]) point to possibility of this step leading to a failure to detect some
of the necessary conditions. Furthermore, recent methodological advancements in
settheoretical thinking include a technique called “NCA” for identifying relationships of
necessity that can make both statements in kind and in degree, thus making full use of
variation in the data [
26
]. The degree of necessity is measured in terms of effect size
(i.e. area of emptiness in the top right corner of the X-Y plot). A comparison of the
results of NCA and QCA [
14
] highlighted the advantages of NCA identifying more
single necessary conditions than QCA, moreover also specifying the degree of
necessity as a clear advantage. In line with these developments and for the purposes of this
paper, we complement QCA with NCA in deriving a maturity model (steps discussed
in section 5).
3
Maturity Models
Maturity models are organizational tools that facilitate internal and/or external
benchmarking while also showcasing future improvement and providing guidelines
through the evolutionary process of organizational development and growth [
27, 28
].
The term “maturity” is defined as “the state of being complete, perfect or ready” [27].
A maturity model usually consists of a sequence of maturity stages [
29
], mostly four
or five [
30
]. Each stage expects the entity (people, process, technology, organisation
etc.) under maturation to fulfil certain requirements that constitute that particular stage
[
31
]. Usually, this is determined by defining critical success factors and boundary
conditions. The critical success factors as prescribed by the maturity model also mean
better outcomes and thus higher business benefits (value) as the organization
progresses on the path to increased maturity. In general, maturity assessment is
understood as a “measure to evaluate the capabilities of an organization”[
29
], with an
underlying assumption of a single linear path to maturity as shown in Figure 1.
From Figure 1, it is evident that without satisfying the boundary conditions, an entity
cannot progress further irrespective of satisfying all other conditions. For example, in
the case of intranet maturity models [
19
], active support of a technology champion or
a sponsor from the top management team is a boundary condition to progress from
stage 1 to stage 2. By formulating boundary conditions as necessary conditions, we
can infer that “the absence of the necessary conditions guarantees failure in terms of
progression to the next stage of the maturity model”. Thus, adopting the set-theoretic
thinking, we postulate stage boundary conditions as necessary conditions and list our
propositions:
P1a: Stage boundary conditions are subsets of critical success factors and can be
identified as necessary conditions.
P1b: Once identified as Stage boundary conditions, their degree of necessity can be
paired with the outcome to derive maturity stages.
Furthermore, as highlighted in Figure 1, a review of extant literature on maturity
models reveals the predominant idea of a single path to maturation (i.e. something
better, advanced, higher performance) mostly linear, forward moving (rarely
regressing), in which the entity improves considerably in terms of desired results i.e.
capabilities, value creation, performance, etc. While notion of maturity has been criticized
widely by King and Kraemer [
33
], Pöppelbuß [
34
] and indirectly by Cleven, Winter
[
35
], Vlahovic [
36
] and many more, there have not been any solution proposed till
date. In this paper, we propose that by applying QCA, we can provide multiple
configurations, translated as multiple paths to maturity. In set-theoretical terms, we adopt
the notion of “equifinality” i.e. an entity or system can reach the same outcome from
different initial conditions and through many different paths [
10
] and list our final
proposition:
P2: A Boolean minimization solution of the Critical success factors (CSF’) would
yield multiple configurations to move from one stage to another, finally reaching full
maturity.
Once the above propositions are empirically validated, i.e. conceptualizing stage
boundaries as necessary conditions and multiple paths to maturity using the logic of
sufficiency, we finally combine the above to inductively derive a maturity model. In
this section we have formulated three propositions and in the next section we explain
our dataset, followed by a demonstrating our approach.
4 Dataset: Selection of Social Media Maturity and Assumptions.
This study uses a dataset of organizations measured for digital maturity by Networked
Business Initiative (NBI). NBI measured digital maturity of organizations in Denmark
in terms of five digital technologies (i.e. social media, web, cloud, data analytics and
mobile) and 6 business functions (PR, Sales & Marketing, Services, HR, R&D and
Leadership). The full description of the data and access to the benchmarking tool is available via the NBI website (www.networkedbusiness.org). The data was collected through a cross-sectional survey whose primary purpose was comparative benchmarking of participating organizations in Denmark.
The design of the cross-sectional survey was open ended, wherein the respondents
were free to choose any technology(s) and any business function(s). This open ended
design with 5 technologies and 6 business functions had possibility of having 1953
unique combinations. NBI measured around 300 organizations, with over 345
respondents over a period of 5 months (October 2015 to March 2016). Given the open
ended design of the survey and not enough respondents, the challenge was to identify
data for analysis. In order to tackle this challenge, we scanned the literature in the
visualization community [
22, 23
] and decided to apply Upset [
21
]. Upset [
21
] is a
technique for the quantitative analysis of sets and their intersections,, with a
capability of handling combinatorial explosion of the number of set intersections. Moreover,
Upset is web-based and open source, thus making is very accessible to us. Given these advantages of Upset, we used the web tool (http://vcg.github.io/upset) to select data from the NBI dataset as shown in figure 2.
After inspecting the Upset visualization (figure 2), we selected data for a single tech
nology (i.e. social media) and its impact on 2 business functions (i.e. PR, Sales and
Marketing). From the 134 data points, post cleaning we were able to identify 85 data
points worthy for further analysis. The detailed descriptive statistics can be found in
appendix 1 & 2. Given the page constraints, we do not go into the depth of the dataset
(i.e. social media maturity across PR, Sales & Marketing), but list out the assumptions
briefly:
1. The social media maturity dataset consists of 14 critical success factors (CSF’s),
also known as conditions (X’s) in set theoretic terminology.
2. Business value realized in PR and Sales and Marketing is the outcome (Y). We
used the average, which is an accepted practice.
3. As shown in Figure 1, “social media maturity ∝ Business value”, this means
higher the social media maturity of an organization, better or higher outcomes or
business value.
4. Critical Success Factors (CSF’s) identified as “necessary but not sufficient
conditions” would be the “stage boundary conditions”.
5. Critical Success Factors (CSF’s) identified as “sufficient but not necessary”
would be another path to maturity.
6. Critical Success Factors (CSF’s) identified as “both necessary and sufficient” would be termed as “most important condition” that an organization must possess them irrespective of which maturity stage there are in.
Now that we have discussed the dataset, assumptions and our propositions, in the next
section, we explain and demonstrate our proposed approach on the social media
maturity dataset.
5 Demonstration of the Proposed Approach
After a detailed review of guidelines and procedures for developing maturity models
[
37-39
], the guidelines for standard practices in QCA [
1, 4, 24, 25
], guidelines for
NCA [
26, 40
] and Fuzzy logic [
16
], we propose the following 6 steps to design an
empirically driven set theoretical maturity model (Figure 3 - Appendix 3). Next we
briefly explain each step using the NBI social media maturity dataset.
Step1- Define the Attributes/Variables (CSF’s): Define the CSF’s, outcome variables
and macro conditions along with the scales used for measurement. Explain the
calculation if and when multiple items are used to measure the CSF (Appendix 1).
Step2- Determine Degree of necessity & Boundary Conditions: by using Necessary
condition analysis [
14, 41
], calculating the effect size and constructing the bottleneck
table (refer [
32
]). In the case of social media maturity, we identified 5 single
necessary conditions and one “necessary and sufficient” condition (Appendix 2). However,
after studying the scale of measurement, Employee empowered culture (EEC) was
determined as a not necessary condition.
Step3 & 4-Fuzzification i.e. rules for Set Membership & propose maturity stages:
Fuzzification also popularly known as “calibration” is a crucial step in QCA requiring
the researcher to assign set membership scores to both outcomes (Y) and conditions
(X). Here the researcher needs to establish qualitative crossover points [
3, 24
] to
assign membership to particular sets. QCA and fuzzy logic scholars [
4
] have proposed
a taxonomy of calibration scenarios [
1, 16
]. We adopt the logistic transformational
assignment (Eq1) as proposed by Ragin [
3
] and Theim et.al [
1
] for assigning full
exclusion, full inclusion and crossover points.
( , ∀[… ], , ) =
Where x is the variable to be
transformed, ∀ex is full exclusion from
the set, ∀cr is the cross-over point
and ∀cr is full inclusion, p and q are
for controlling the shape of the
membership function.
1 −
1
[
2 ∀
∀ −
− ∀
1
[
2 ∀
∀
−
− ∀
]
]
0
1
If ∀ex ≥ xi,
If ∀ex< xi
≤ ∀cr,
If ∀cr< xi ≤
∀in,
If ∀in< xi
……..Eq1
Our primary interest in this step was defining the maturity stages in terms of set
memberships, which we measured through a proxy of business value realized (Y).
Following the configurational approach [
10, 24
], we also created fuzzy set measures
of above-average business value realized (i.e. set with high maturity). This
“benchmark” of above-average was set at 50% business value realized (i.e. score of 2). The
reasoning was equally motivated by calibration of survey data for QCA [
12
] and
qualitative reasoning among the authors that if an organization has derived “at least
high value” in either PR or Sales & Marketing (2.5 and above), then it is more in the
set of high maturity. For this first set, we coded full inclusion of ∀
= 0.5 and full
exclusion of ∀
= 3.5 with a cross over point of ∀
= 2.1. As highlighted in Figure
3 (High Maturity), an organization with business value less than 2 is “more out than
in”, while business value more than 2 is “more in than out”. The second set was
organizations with very high business value realized (i.e. Very High maturity). Here the
crossover point was raised to 3, while full exclusion for the higher end point was set
at 4. Finally, in order to examine what configurations lead to low business value
realized, we created measures of membership not-high and low business value realized.
This third set was simply coded as the negation of the set with high maturity
(Appendix 3), with a full exclusion of 2.5 and 0, with a cross over at 1.5. Following the
fuzzification of the Outcome (Y), the conditions (CSF’s) are now fuzzified or calibrated
using both the empirical evidence at hand and qualitative interference. For example,
FTE (measured as 0 for none, 1 for part time resource, 2 for one resource, 3 for two or
more) was coded a full exclusion of 0 and 3, with a crossover of 0.9, indicating that at
least a part time resource (i.e. score of 1) is required for an organization to achieve
high maturity. Other CSF’s were similarly coded and the inclusion, exclusion and
cross over points have been listed in appendix 1.
Step5 - Fuzzy Inference System based on Qualitative Comparative Analysis:
Inferencing is a process to “evaluate all pre-defined rules to perform the reasoning process”
[
16
]. In our case, we employ the pre-defined rules of Qualitative Comparative
Analysis [
1, 2
] to first convert the fuzzy sets into crisp truth table values, then employ
Boolean minimization to arrive at final solution1. Steps 3, 4 and 5 work in an iterative
cycle as illustrated in Figure 3 (Appendix 3) until an optimal solution is obtained in
what Ragin [
3
] terms as an “analytical moment”. This iterative cycle might also lead
to formulations of some macro conditions, improved case and theoretical knowledge.
1 Refer 1. Thiem, A. and A. Dusa, Qualitative comparative analysis with R: A user’s guide. Vol.
5. 2012: Springer Science & Business Media. Page 54 – 79 for detailed steps.
In our case, we dropped digital strategy (DS) as it did not contribute to achieving a
solution, and created two macro conditions. The first macro condition termed “FUE”
was combination of common necessary conditions (Appendix 2) required for high and
very high maturity stages. The second macro condition “IT Policy (ITP)” was arrived
through what Ragin [
3
] terms “colligations”, i.e. meaningful collections of facts or
evidence. The logic for the macro conditions is described in table 2. The next logical
step was to employ the prescribed steps[
1, 42
] for QCA. We set the inclusion criteria
of 0.72 and tested the final the configurations for paradoxical relations [
1
]. Post this
analysis, we found 3 configurations for high maturity stage and one configuration for
the low maturity stage, but none for very high maturity stage.
Step 6 – Visualize and present the maturity logic: The sixth and final step was
visualizing the set theoretical maturity model and assessment logic for the future. There
were multiple options suggested in literature to present the results [e.g.
CorePeriphery configuration chart [
24
], Solution as Boolean expression [
1, 3
],
Relevancetrivialness table [
25
]]. We considered all these options and chose the Core-Periphery
configuration chart, given its visual symmetry with prior maturity models. Figure 4
shows the results for high maturity stage and low maturity stage respectively. From
the configurations, it is possible to present the maturity logic as a set of fuzzy rules
[
16
]. Some of the rules are as follows:
1. IF ESC is less than two THEN maturity is LOW.
2. IF ESC is more than one and FTE is zero THEN maturity is LOW
3. IF ESC is more than one and Extent of Use high and FTE is at least one and
management support is high and explorative culture is present THEN
maturity is HIGH.
4. IF ESC is more than one and Extent of Use high and FTE is at least one and
management support is high and explorative culture is present and
Investment is increasing THEN maturity is definitely HIGH and may be VERY
HIGH.
With the current dataset, while we have established boundary conditions for
progressing towards very high maturity, we can only speculate about the configurations in the
very high maturity stage. Therefore, the fuzzy rules would call for qualitative
interference or collection of more data to determine if an organization is in the very high
maturity stage. Another possibility is the use the max-membership principle [
16
] or
the concept of misfit [
43
] to assess an organizations maturity.
Black circles indicate presence of a condition; circles with “X” indicate its absence.
Large circles indicate core conditions; small ones indicate peripheral conditions.
Blank spaces indicate “don’t care”, i.e. presence or absence has no impact [
24
]
Figure 4: Visualisation of set theoretical social media maturity configuration.