=Paper=
{{Paper
|id=None
|storemode=property
|title=Quality Evaluation for Strategic Alignment Engineering: An eGovernment Application
|pdfUrl=https://ceur-ws.org/Vol-599/BUISTAL2010_Paper1.pdf
|volume=Vol-599
}}
==Quality Evaluation for Strategic Alignment Engineering: An eGovernment Application==
https://ceur-ws.org/Vol-599/BUISTAL2010_Paper1.pdf
Quality Evaluation for Strategic Alignment
Engineering:
An eGovernment Application
Pei Li and Gianluigi Viscusi
Department of Informatics, Systems and Communication (DISCo)
University of Milan - Bicocca
{pei.li, viscusi}@disco.unimib.it
Abstract. This article explores the role of quality evaluation to measure strate-
gic alignment and support strategic IT requirements. In particular, the focus is
on eGovernment context that presents a set of characteristics asking for different
evaluation metrics from the ones adopted for business in the private sector (usu-
ally more investigated in terms of IT strategy alignment). The proposal approach
implements a subset of a quality framework on a real scenario for the purpose
of quality evaluation. It exploits real-life data from interviews and questionnaires
filled by a first sample of non-EU foreign researchers and students in the scope
of the Computer Science department where the authors are affiliated, who have
applied their residency permits within the past three years. Finally, a probability-
based analysis is carried out in order to elicit quality dimensions with priority
among the ones considered.
1 Introduction
In this article, we propose to explore the role of quality evaluation to measure strategic
alignment and support strategic IT requirements. In particular we focus on the eGov-
ernment context that presents a set of characteristics asking for different evaluation
metrics from the ones adopted for business in the private sector which are usually more
investigated in terms of IT strategy alignment. Indeed, it is worth noting that, in the
context of eGovernment, strategy is not oriented to obtain a competitive advantage and
shareholder value as in the private sector; on the contrary, eGovernment value heavily
depends on political and social objectives, such as trust in government, social inclu-
sion, community regeneration, community well-being and sustainability [1]. In general
terms, in the context of eGovernment, value relies mainly on pubic value. As stated by
Grimsley and Meehan [1] on the basis of Moore’s conceptualization [2], public value
can be interpreted as the value that citizens and their representatives seek in relation to
strategic outcomes and experiences of public services.
Due to these issues, we need to consider value not only from an economic per-
spective, but also as the degree to which public policies improve the quality of life
of constituencies - namely citizens and businesses - by improving the quality of pub-
lic services and of the public administration organization and processes. Therefore, a
systemic perspective on quality is required. Among the available frameworks for eGov-
ernment projects assessment [3], in this paper we consider GovQual [4] for its focus
M. Petit, G. Gal, A. Castiaux, J. Ralyté, and P. Plebani (Eds.):
CAiSE 2010 Workshop BUSITAL’10, Hammamet, Tunisia, pp. 1-15, 2010.
2 G. Viscusi and P. Li
on quality dimensions and their mutual relationships for the different facets involved
in eGovernment projects (i.e. economic, social, organizational, legal, and technological
facets).
In this paper we consider a scenario focused on the Italian context, in which the cho-
sen quality dimensions result from a preliminary analysis of online available eGovern-
ment strategy documentations and we assume that they implement the political-vision-
related choice; while the scenario process is modelled on the basis of direct observation
over a one-year period from September 2008 to September 2009. The corresponding
quality evaluation exploits real-life data obtained from interviews and questionnaires
filled by a first sample of non-EU foreign researchers and students in the scope of our
Computer Science department, who have applied their residency permits within the past
three years. Finally, a probabilistic simulation and analysis based on the sampled data
have been carried out in order to elicit quality dimensions with priority among the ones
considered.
The paper is organized as follows. Section 2 discusses related work. Section 3 de-
scribes the scenario for the application of the proposed approach, with a focus on the
Italian context. Specifically, the considered administrative process is detailed in Section
3.1, while qualities considered in the scenario are modelled in Section 3.2. Finally, a
prior computation for quality dimension evaluation is detailed in Section 3.3, based on
which we present probabilistic analysis with Monte Carlo simulation in Section 3.4.
Section 4 states conclusions and future works of the paper.
2 Related Work
IT Strategy alignment has been considered as a major issue for management of informa-
tion systems [5, 6] in order to provide competitive advantages to IT enabled businesses.
Strategic IT requirements engineering [7] aims to model alignment in order to elicit
strategy related requirements, the most appropriate (among other facets) for the opera-
tional level of the considered domain of intervention [8]. To this end business modelling
represents an emerging effort providing a set of frameworks for an explicit link between
system requirements and the objectives of business strategy [9].
The most comprehensive and well-defined languages and frameworks for business
modelling are the Resource-Event-Actor (REA) framework, the e3value framework, and
the Business Model Ontology (BMO). The REA framework [10] is focused on represent-
ing increases and decreases of value in organizations, which has its origins in business
accounting. The core concepts in REA are Resource, Event, and Actor, where every
transaction can be described as two events in which two actors exchange resources.
e3value framework [11] explicitly focuses on resources exchange as value objects. Ba-
sic concepts in the e3value ontology are actors, value objects, value ports, value inter-
faces, value activities and value exchanges. The BMO [12] provides a framework that
consists of nine core concepts classified under four categories, as described in the fol-
lowing: the category product as a single concept, that is value proposition; the category
customer interface has three concepts, namely target customer, distribution channel,
and relationship; the category infrastructure management has three concepts, namely
Quality Evaluation for Strategic Alignment Engineering 3
value configuration, capability, and partnership; the category financial aspect has two
concepts, namely cost structure, revenue model.
Besides these frameworks which support a value-based documentation of the busi-
ness model to be implemented in the systems, other models and approaches have been
proposed in the literature, focusing more on strategies to model and measure the align-
ment as a source for requirement engineering, such as the Business Rules Group Mo-
tivation Model (BRG-Model) [13], the B-SCP framework [14], INSTAL method [15]
(based on the Map Model [16] and exploiting quality metrics [17]), and the e3alignment
model [18] as an extension of the e3value framework.
Considering now the operational level, i* [19] allows to model organizational goals
and tasks together with actors and resources. Business process modelling is necessary
as a representation of how the operational level works and is managed - in this paper we
consider administrative processes as business processes - by means of the state of the
art tools and concepts such as the Business Process Modelling Notation [20]. In partic-
ular, process models are the basis of the development of specific workflow applications.
The models describe the process structure and logic on a type level, whereas workflow
applications support the execution of single processes on the instance level.
In this paper we discuss a quality evaluation framework in order to explore its role
in providing metrics which support the alignment of operational level with the strategy
to be implemented in the eGovernment initiatives. Moreover, we focus on operational
level representation by means of process models. Whereas is relevant that we discuss
here the state of art models and frameworks for business modelling, this topic will be
considered in future work.
3 Residency Permit for Study Reason: A Scenario
In this section, we use the residency permit application process as a scenario to eval-
uate strategic alignment with respect to quality dimensions and to help analyzing rela-
tionships among them, in order to identify evolution requirements that lead to a better
strategic alignment. The scenario is focused on the Italian context. According to the
agreement signed by the Italian Ministry of Internal Affairs and Poste Italiane SPA (the
major Italian company for postal services and mail), non-EU foreign students should
present applications for residency permits in Italy annually. The residency permit for
study reasons may only be applied and renewed if the entry visa was issued to attend a
study course of more than one year in length. The first application, valid for one year,
should be made within 8 days upon applicants’ arrival in Italy and renewed annually.
But in any case, it cannot be renewed for more than three years above the duration of
the study course. In the following, we first introduce a workflow system, METEOR,
to model the residency permit application process, and then two quality dimensions,
efficiency and effectiveness, are mainly modelled w.r.t. different layers in GovQual
framework; based on which we present quality computation methods and finally we
use Monte Carlo simulation tools to analyze dependencies among quality dimensions.
4 G. Viscusi and P. Li
3.1 Process of Applying Residency Permit for Study Reason
Workflow Structure Existing workflow systems provide a set of indispensable func-
tionalities that manage and streamline business process. Yet, few research groups have
concentrated their efforts on enhancing workflow systems to support workflow Quality
of Service management. Most of the research carried out in order to extend workflow
systems’ capabilities to include project management features has mainly been done for
the time dimension [21–23]. In this paper, we adopt a workflow management system
named METEOR [24] in which, not only is time considered, but also other dimensions
associated with workflow executions, such as cost and efficiency. To compute quality
of service dimensions, METEOR has developed both a model for the specification of
workflow QoS and methods to analyze and monitor QoS, which can be exploited to
evaluate different quality dimensions in the following scenario.
As defined in [24], a workflow is composed of tasks and transitions. Tasks are rep-
resented using a circle, networks (sub-workflows) using rounded rectangles, and tran-
sitions are represented using an arrow. Transitions express dependencies between tasks
and are associated with an enabling probability (p1 , p2 , .., pn ). When a task has only
one outgoing transition, the enabling probability is 1. In such a case, the probability can
be omitted from the graph.
Residency Permit Workflow Description Residency Permit process is composed of
four sub-processes (sub-workflow): Document Preparation, Application Identification,
Fingerprint Collection, and Permit Release. Detailed process goes as follows (seen in
Figure 1):
– In the stage of Document Preparation, interested parties (non-EU foreign students)
ask for related application kit available at certain post offices and fill the forms.
Signed application forms together with other required documents must be sent by
post to the Police Office (Questura) of their province of residence for the first issue
of residency permit. A collection of documents include:
• Complete photocopy of passport or other equivalent document;
• Photocopy of the statement certifying the course of study to be taken, certified
by the Italian Diplomatic/Consular mission when the entry visa is issued;
• Photocopy of insurance policy, valid throughout the country and for the entire
period of validity of the residency permit, against the risk of illness and injury;
• Photocopy of the documentation certifying to the availability of adequate fi-
nancial resources for the validity period of the residency permit.
– In the stage of Application Identification, related immigration offices in Italy will
check the completeness and correctness of received application documents, and
publish the identification result on Internet, where applicants can query their appli-
cation statements by a user ID and password assigned to the application receipt;
– In the stage of Fingerprint Collection, once application documents are identified, in
about three months or even longer applicants will receive a registered letter from the
Immigration Office summoning them to be fingerprinted (the same information is
sent to applicants by SMS as well). Meanwhile, each applicant can inquire his/her
application statement physically to related immigration offices according to their
application statements published on Internet.
Quality Evaluation for Strategic Alignment Engineering 5
Fig. 1: Business Process of Applying ”Residency Permit for Study Reason”
– In the stage of Permit Release, after the collection of fingerprint, applicants will
have another appointment (the same information is sent to applicants by SMS as
well) to release their regular residency permit. In other cases, they will get tempo-
rary permits valid for three months or no permit.
3.2 Quality Modelling
As shown in Figure 2 (text in bold), in this paper we mainly focus on the assessment
of a limited set of quality dimensions: efficiency, which can be mutually influenced by
other dimensions, such as effectiveness. Concerning efficiency, we distinguish between
temporal, economic, and procedural efficiency. Temporal efficiency refers to efficient
use of time in service production and provision. Temporal efficiency is significant in
two layers. At the process/service layer it is classified in terms of two dimensions:
the user time (the average time spent by users to obtain the service) and the service
provision time (the average time spent by organizational units to produce the service).
Temporal efficiency at the ICT infrastructure layer results in response time, the usual
dimension considered for hardware and network infrastructures. The measure is simply
the time span that the resources need to execute transactions and respond to queries.
Economic efficiency concerns the costs sustained for service provision and their trends
in time.
The procedural efficiency concerns the level of bureaucratic simplification of the
administrative processes, meaningful for the organizational system layer. Procedural
6 G. Viscusi and P. Li
Fig. 2: Quality Dimensions in different layers of GovQual Framework
efficiency refers to the obligations and constraints that laws impose on the administra-
tive processes and on the interactions between administrations and citizens. We asso-
ciate procedural efficiency with a metric consistent in the level of simplification. This
is measured in terms of the number of interactions required by users to provide useful
information in order to complete the service.
On the other hand, effectiveness is meaningful at the process/service layer and at
the data layer. At the process/service layer it can be expressed in terms of the achieve-
ment of the users’ expectations. At the data layer, effectiveness results in two dimen-
sions: accuracy and completeness. In this paper we evaluate completeness in the data
layer. We use both qualitative (user comments) and quantitative (number of complaints)
indicators as metrics for effectiveness assessment in process/service layer. User com-
plaints/comments reflect the users’ perception of the accuracy and reliability of service
provision, and the evaluation of the service-oriented attitude of personnel in charge
of the service within organizational units. Table 1 shows a mapping from qualitative
indicators (user comments) to quantitative indictors (number of complaints). Data com-
Table 1: Effectiveness Assessment
User Comments Number of Complaints
Unacceptable > 16
=
Poor >8
=
Satisfactory >4
=
Good >2
=
Perfect >0
=
Quality Evaluation for Strategic Alignment Engineering 7
pleteness is the extent to which data is of sufficient breath, depth and scope for the task
at hand. In most service execution processes, as in the residency permit scenario, the ex-
ecution is incapable to proceed if data is not completely provided. In this sense we mea-
sure data completeness with a boolean < yes, no > domain. According to the above
definitions of efficiency and effectiveness in different layers of GovQual framework, we
can specify temporal and procedural efficiency, process/service and data effectiveness
in residency permit application workflow as follows (Table 2):
Table 2: Current Values for Different Quality Dimensions in Residency Permit
Workflow7
Quality Dim. Layer Task Current Value Prob. Dist.
User Time Process/Service Documents Preparation 1 - 2 days N (1.5, 0.03)4
Application Kit Preparation 0.5 - 1 day N (0.75, 0.01)
Application Delivery 3 - 5 hours N (4, 0.11)
Application Statement Inquiry3 3 months 1
Appointment Notification (1)1 2 - 13 months H(2, 13)4
Fingerprint Collection (1)2 20 - 30 days N (25, 2.78)
Regular Permit Release 20 - 30 days N (25, 2.78)
Service Provision Process/Service Application Kit Preparation 1 - 5 mins N (3, 0.45)
Time Application Delivery 1 - 2 days N (1.5, 0.03)
Application Check 3 - 7 days N (5, 0.45)
Application Statement Inquiry3 1 - 2 hours N (1.5, 0.03)
Temporary Permit Preparation 1 - 2 hours N (1.5, 0.03)
Fingerprint Collection(1)(2)2 0.5 - 1 hour N (0.75, 0.01)
Appointment Notification (2)1 0.5 - 1 hours N (0.75, 0.01)
Temporary Permit Release 1 - 2 hours N (1.5, 0.03)
Regular Permit Release 0.5 - 1 hours N (0.75, 0.01)
Response Time ICT Infrastructure Application Statement Release 1 - 3 hours N (2, 0.11)
Appointment Notification(1)(2) 1 - 2 days N (1.5, 0.03)
Level of Simp. Organization System Application Statement Inquiry in Section 3.45 in Section 3.45
Effectiveness Process/Service Residency Permit Application in Section 3.46 in Section 3.46
Completeness Data Document Preparation Yes 0.5
No 0.5
Application Kit Preparation Yes 0.5
No 0.5
1. Appointment Notification (1) refers to the first appointment for fingerprint collec-
tion and original document check; Appointment Notification (2) refers to the sec-
ond notification for regular permit release.
2. Fingerprint Collection (1) refers to the fingerprint collection and original document
check for regular permit processes, while Fingerprint Collection (2) refers to the
fingerprint collection and original document check for temporary permit processes.
8 G. Viscusi and P. Li
3. Since there are internal loop transactions within the task of Application Statement
Inquiry (Figure 1), in Table 2 we only consider time spent for a single task of
Application Statement Inquiry.
4. Probability distribution of each task is assigned on the basis of a few interviews.
Currently we consider two distributions, normal distribution N (µ, σ 2 ) and his-
togrm distribution Histogrm(min, max, {p1 , p2 , ..., pn }) (in this scenario, Appoint-
ment Notification (1) follows a distribution of Histogrm(2, 13, {0.5, 2.5, 2, 1})).
In the future stage, we plan to present more precise distributions by exploiting sur-
veys over larger amount of application cases. In addition, due to the complexity
of relationships among quality dimensions, for certain dimensions we will present
their joint distributions instead of marginal distributions, i.e., joint distribution of
process/service effectiveness and efficiency.
5. Level of Simplification of residency permit is dependent from other quality dimen-
sions. Since interactions are fixed for other tasks in the workflow, the most flexible
part is related to the task of Application Statement Inquiry. Considering that ap-
plicant/staff interactions for Application Statement Inquiry are subjective to appli-
cants, we fix the frequency for inquiry in this scenario, which is also reasonable and
practical in real life: suppose in the ideal case, applicants will get appointment noti-
fications (both by post and by SMS) in 3 months after their applications have been
identified. If not the case, applicants will wait for another 3 months and inquiry
their application statements at related Immigration Offices physically for the first
time, and once every 3 months if still necessary. Since in 12 months after the first
application - which means maximally three interactions for application statement
inquiry - applicants are obliged to renew their permits, the workflow will result in
the following consequences: within one interaction, applicants may get their regu-
lar permits of stay; after two interactions, applications may get temporary permits;
after three interactions applications may get no permits. Distributions of number of
interactions and permit types will be analyzed in Section 3.4 where we observe a
strong correlations between the two factors.
6. Effectiveness in Process/Service layer is presented by a probability distribution of
user complaints (also considered as user comments) with respect to different re-
sults they get at the end of permit application workflow. Distribution of this quality
dimension and the factors which have significant impacts on it are deduced from
several interviews we made together with Monte Carlo simulations in Section 3.4.
7. Two simple observations can be achieved from Table 2. Different quality dimen-
sions influence each other in two levels: (i) within single task among the same/different
layer(s); (ii) over the whole service process among the same/different layer(s). A
more statistic analysis of the above observations will be presented in Section 3.3.
3.3 Prior Computation for Quality Dimension Evaluation
In this section we compute overall time used for residency permit workflow, in order
to evaluate relationships among different quality dimensions, i.e., efficiency and effec-
tiveness, and to optimize quality requirements. Overall time for the whole workflow
can be computed in three steps: (i) first we present formulas to calculate time spent for
each atomic task; (ii) based on which we reduce complex task structures, including (1)
Quality Evaluation for Strategic Alignment Engineering 9
sequential, (2) parallel, (3) conditional, (4) fault-tolerant, (5) loop, and (6) network, into
atomic ones; in the following we show three complex structure reduction examples, for
Document Collection, Application Kit Collection and Application Statement Inquiry,
respectively; (iii) finally, according to the above modelings, we present the overall time
spent for residency permit workflow in a more abstracted level.
Atomic Task Time Computation In Table 3, computation details for all tasks in-
volved in residency permit application is presented, in which we assign an task ID, ti ,
to each task in the workflow. For each task ti in a service, the overall task time T (ti ) is
computed as the sum of user time U T (ti ) and/or service provision time P T (ti ) and/or
Response Time RT (ti ), all of which are considered as the main indictors to evaluate
efficiency of a service. The overall task time is also effected by other quality dimensions
in different layers. Relationships among them will be the focus of the next sections.
Table 3: Atomic Task Time in Residency Permit Application Workflow
ID Task Task Time
t1 Document Preparation User Time
< 1 − 2 > days
t2 Application Kit Preparation User Time + Service Provision Time
< 0.5 − 1 > day+ < 1 − 5 > mins
t3 Application Delivery User Time + Service Provision Time
< 3 − 5 > hours+ < 1 − 2 > days
t4 Application Check Service Provision Time
< 3 − 7 > days
t5 Application Statement Release Response Time
< 1 − 3 > hours
t6 Application Statement Inquiry User Time + Service Provision Time
3months+ < 1 − 2 > hours
t7 Appointment Notification (1) User Time + Response Time
< 3 − 6 > months+ < 1 − 2 > days
t8 Temporary Permit Preparation Service Provision Time
< 1 − 2 > hours
t9 Fingerprint Collection (1) User Time + Service Provision Time
< 20 − 30 > days+ < 0.5 − 1 > hour
t10 Fingerprint Collection (2) Service Provision Time
< 0.5 − 1 > hour
t11 Temporary Permit Release Service Provision Time
< 1 − 2 > hours
t12 Appointment Notification (2) Service Provision Time
< 0.5 − 1 > hour
t13 Regular Permit Release User Time + Service Provision Time
< 20 − 30 > days+ < 0.5 − 1 > hour
10 G. Viscusi and P. Li
1. Since applicants will get Appointment Notification (2) for regular permit release at
the end of their fingerprint collection task, we mainly consider the service provision
time as time used for completing this task. Meanwhile, applicants will get the same
reconfirmation information by SMS, which is measured by Response Time in ICT
Infrastructure layer. However we consider such a process structure as a fault tol-
erant sub-workflow, and for the sake of simplicity, response time for Appointment
Notification (2) is kept in Table 2 but omitted in the overall time calculation.
2. Fingerprint Collection (2) for temporary permit processes is executed at the end of
Application Statement Inquiry task, thus we mainly consider service provision time
as time spent for completing this task.
Complex Task Time Computation In the following we present complex structure re-
duction computations for three atomic tasks respectively: Document Collection, Appli-
cation Kit Collection and Appointment Statement Inquiry (Figure 3). The loop structure
(a) Document Collection (b) Appl. Kit Collection
(c) App. Statement Inquiry
Fig. 3: Complex Task Time Computation
of t1 Document Preparation is influenced by data completeness in Data layer. Let tl1
be the reduced loop task of t1 , then time spent for completing tl1 , denoted as T (tl1 ), is
calculated as follows:
T (tl1 ) = T (t1 )(1 + f1 (d)) (1)
In which, f1 (d) is the frequency of document preparation and is inversely proportional
to data completeness:
(
0 DataCompleteness := Y es
f1 (d) = (2)
1 DataCompleteness := N o
Since data completeness for t1 is uniformly distributed (see Table 2), its probability
distribution P1 is: (
0.5 f1 (d) = 0
P1 (d) = (3)
0.5 f1 (d) = 1
Quality Evaluation for Strategic Alignment Engineering 11
Similarly for Application Kit Preparation t2 , of which we will not present details due
to the reason of space.
As for Application Statement Inquiry t6 , let tl6 be the reduced loop task of t6
and random variable X as the number of interactions in fulfilling t6 , in which {x =
0, 1, 2, 3} ∈ X. T (tl6 ), denoted as time spent for completing tl6 , the task is calculated
as follows:
T (tl6 ) = xT (t6 ) (4)
As similarly to t1 and t2 , t6 influences different quality dimensions. However the impact
is not only within single task but also involved a higher abstract level, in this scenario as
Fingerprint Collection sub-workflow. Therefore we will focusing analyzing probability
distributions in Fingerprint Collection sub-workflow in the following sections.
Sub-Workflow Time Computation Based on the above modelling, we abstract tasks
involved in the residency permit workflow into four sub-workflows: Document Prepa-
ration, Application Identification, Fingerprint Collection, and Permit Release. We first
compute time spent to fulfill each sub-workflow, and use the results for final overall
time calculation.
As shown in Figure 1, time spent for the sub-workflow of Document Preparation,
denoted as T (sw1 ), is computed as follows:
T (sw1 ) = max{T (tl1 ), T (tl2 )} + T (t3 ) (5)
Time spent for the sub-workflow of Application Identification, denoted as T (sw2 ), is
computed as follows:
T (sw2 ) = T (t4 ) + T (t5 ) (6)
Before computing time spent for the sub-workflow of Fingerprint Collection, let’s first
recall that random variable X is defined as the number of interactions in fulfilling t6 ,
in which {x = 0, 1, 2, 3} ∈ X. Similarly, we define random variable Y as permit types
applicants will get at the end of residency permit application workflow, and random
variable Z as user complaints / comments.
1 RegularP ermit
Y = 0 T emporaryP ermit (7)
−1 N oP ermit
2 P erf ect
1 Good
Z= 0 Satisf actory (8)
−1 P oor
−2 U nacceptable
Time spent for Fingerprint Collection sub-workflow, denoted as T (sw3 ), is computed
as follows:
T (t7 ) + T (t9 ) {y = 1}
T (sw3 ) = T (tl6 ) + T (t8 ) + T (t10 ) {y = 0} (9)
T (tl6 ) {y = −1}
12 G. Viscusi and P. Li
The time spent for the sub-workflow of Permit Release, denoted as T (sw4 ), is computed
as follows:
T (t12 ) + T (t13 ) y = 1
T (sw4 ) = T (t11 ) y=0 (10)
0 y = −1
Finally, the overtime spent for completing the whole workflow of residency permit,
denoted as T , is computed as follows:
T = T (sw1 ) + T (sw2 ) + T (sw3 ) + T (sw4 ) (11)
In the next section, we will use the above quality modelling results as input for Monte
Carlo simulation, in order to get a deeper and statistical analysis of quality evaluation
for strategic alignment, and to further improve requirement re-engineering in business
processes.
3.4 Adopting Monte Carlo Simulation for Quality Assessment
With the residency permit scenario modelled into the above workflow and sampled real-
life data at hand, the idea of adopting Monte Carlo simulation for quality evaluation is
inspired with no surprise. Specifically, the simulation iteratively runs over the workflow
which imitates real-life residency permit application processes, where we can benefit
from two aspects: (i) it overcomes our limited sample size and extends our observation
with user-defined number of iterations; (ii) it enables statistic analysis and predictions
over the underlying scenario which is complex, nonlinear and involves more than just a
couple of uncertain parameters.
Based the above reasoning, we initialize the Monte Carlo simulation scenario (in
Table 4) with our sampled real-life data as input parameters (in Table 2 and 3) and
equations in Section 3.3 as uncertain functions. The expected outputs include a subset
of efficiency and effectiveness indicators we will explicitly discuss in the following text.
Table 4: Monte Carlo Simulation Settings
Simulation Runtime
Number of Iterations 5000
Number of Simulations 1
Data Sampling
Sampling Type Monte Carlo
Generator RAN2I
Initial Seeds Randomely
Quality Evaluation for Strategic Alignment Engineering 13
Simulation Results and Discussion Among several simulation statistics, the one of
our interest is sensibility, i.e., analysis of which input factor having the most significant
impact on the output. Sensibility of the model shows the relationships and dependen-
cies of different quality dimensions of multiple layers in this underlying eGovernment
application. Detailed sensibility statistics is available in Figure 4.
Focusing on temporal efficiency of the residency permit process, in sub-workflow
1 and 2, Data Completeness (Effectiveness in ICT layer) and Service Provision Time
have strong influence on the sub-workflow time. However focusing on the total time
spent for the overall workflow, we discover User Time (in this scenario, user time of
Appointment Notification(1)) affecting the overall temporal efficiency.
On the other hand, the level of simplification of the considered scenario depends on
the number of inquiries applicants made for application statement check, which is af-
fected by the time applicants wait for their first appointment notifications. The analysis
points out that User Time of Appointment Notification (1) has a strong impact on level
of simplification (with correlation coefficient value of 0.949 in the range of [−1...1],
meaning that the longer the user time for appointment notification (1) is, the more in-
teractions the process has). Since level of simplification is a joint distribution of X (the
number of interactions) and Y (permit type), we also simulate the sensibility factor of
the latter (permit type). Resulted data shows that the type of permit an applicant gets
at the end of the process is also affected by user time of the first appointment notifi-
cation (with correlation coefficient value of −0.818, meaning the longer the user time
for appointment notification (1), the permit type resulted from the process can vary
from regular, temporary to no permit released). The above analysis indicates a strong
dependency between X and Y .
Furthermore, sensibility analysis of effectiveness in process/service layer (in the
form of user comments, denoted as Z) is an interesting observation. We categorize
Z according to the overall time spent for residency permit for the following reasons:
(i) overall time spent directly covers temporal efficiency in different layers; (ii) it is
also affected by data completeness; (iii) different types of permit result in different
overall time spent, which in turn is affected by level of simplification in organization
system layer. In another words, the output of overall workflow time can be considered
as a representation of all input quality dimensions in the scenario, and therefore it is
reasonable to assume that sensibility analysis of effectiveness can be observed from the
correlation between overall workflow time and Z. Statistic result in Figure 4 shows that,
among all possible factors, process/service effectiveness is again heavily influenced by
user time (with correlation coefficient value of −0.712, meaning the longer applicants
wait, the less effectively they could benefit from the underlying service).
4 Conclusions and Future Work
In this article, we have explored the role of quality metrics to measure strategic align-
ment and support strategic IT requirements. In particular we have focused on the eGov-
ernment context. To this end we have adopted a framework for systemic quality as-
sessment, considering qualities for the different facets of the subject domain (pro-
cess/service, organizational system, and ICT technologies). Values for different qual-
14 G. Viscusi and P. Li
Fig. 4: Sensibility Statistics of Residency Permit Scenario
ities have been defined on the basis of results from interviews. We have then applied
a probabilistic model to identify relationships and dependencies between the quality
dimensions. In particular, we have identified that i) user time of appointment notifica-
tion has a strong impact on level of simplification and consequently ii) type of permit
an applicant gets at the end of the process and the degree of client satisfaction of the
service scenario are also affected by user time of the first appointment notification. The
obtained results indicate that, from an alignment perspective, the probabilistic analy-
sis together with the quality framework have provided useful information about critical
factors at operational level for improving services related to residency permit, namely
the relevance of user time with respect to the type of permit required. Yet actual gov-
ernment strategy focuses on the improvement of service provision time at the back-end
level.
Limitations of the research are related to (i) the limited set of qualities considered,
(ii) the relative small sample of interviews and questionnaires collected, and (iii) the
lack of a formal model to map intentions at strategy level with modelling of workflow
at operational level. In further works we will exploit a survey approach to extend the
sample size of interviewees for analysis and extend the actual framework by adopting a
formal model for strategy representation.
References
1. Grimsley, M., Meehan, A.: e-government information systems:evaluation-led design for
public value and client trust. European Journal of Information Systems 16 (2007) 134 – 48
Quality Evaluation for Strategic Alignment Engineering 15
2. Moore, M.: Creating Public Value: Strategic Management in Government. Harvard Univer-
sity Press, Cambridge, MA (1995)
3. Fitsilis, P. Anthopoulos, L., Gerogiannis, V.C.: Assessment frameworks of e-government
projects: a comparison. In: PCI 2009 Conference Proceedings. (2009)
4. Batini, C., Viscusi, G., Cherubini, D.: Govqual: A quality driven methodology for e-
government project planning. Government Information Quarterly 26 (2009) 106–117
5. Luftman, J.: Competing in the information age: Strategic alignment in practice. Oxford
University Press, New York (1996)
6. Henderson, J., Venkatraman, N.: Strategic alignment: Leveraging information technology
for transforming organizations. IBM Systems Journal 32 (1993)
7. Babar, A., Cox, K., Bleistein, S.J., Verner, J.M.: Towards evolution of strategic it require-
ments. In: SAC. (2007) 1220–1227
8. Bleistein, S.J., Cox, K., Verner, J.: Strategic alignment in requirements analysis for organi-
zational it: an integrated approach. In: SAC ’05: Proceedings of the 2005 ACM symposium
on Applied computing, New York, NY, USA, ACM (2005) 1300–1307
9. Johannesson, P. In: The Role of Business Models in Enterprise Modelling. Springer (2007)
10. McCarthy, W.E.: The rea accounting model: A generalized framework for accounting sys-
tems in a shared data environment. The Accounting Review 58 (1982) 554–578
11. Gordijn, J., Akkermans, J.M.: Value-based requirements engineering: exploring innovative
e-commerce ideas. Requir. Eng. 8(2) (2003) 114–134
12. Osterwalder, A., Pigneur, Y., Tucci, C.: Clarifying business models: Origins, present, and
future of the concept. Communications of the AIS 2005 16 (2005)
13. Kolber, A.B., Estep, C., Hay, D., Struck, D., Lam, G., Healy, J., Hall, J., Zachman, J.A.,
Healy, K., Eulenberg, M., Fishman, N., Ross, R., Moriarty, T., Selkow, W.: Organizing
business plans: The standard model for business rule motivation (2000)
14. Babar, A., Cox, K., Tosic, V., Bleistein, S.J., Verner, J.M.: Integrating b-scp and map to
manage the evolution of strategic it requirements. Information & Software Technology 50(7-
8) (2008) 815–831
15. Thevenet, L.H., Salinesi, C.: Aligning is to organization’s strategy: The instalmethod. In:
CAiSE. (2007) 203–217
16. Rolland, C.: Capturing system intentionality with maps. In Krogstie, J., Opdahl, A.L.,
Brinkkemper, S., eds.: Conceptual Modelling in Information Systems Engineering. Springer
(2007)
17. Viscusi, G., Thevenet, L.H., Salinesi, C.: Strategic alignment in the context of e-services -
an empirical investigation of the instal approach using the italian egovernment initiative case
study. In: CAiSE. (2008) 163–166
18. Pijpers, V., Gordijn, J., Akkermans, H.: e3alignment: Exploring inter-organizational align-
ment in net worked value constellations. IJCSA 6(5) (2009) 59–88
19. Yu, E.S.K., Mylopoulos, J.: From e-r to ”a-r” - modelling strategic actor relationships for
business process reengineering. In: ER. (1994) 548–565
20. White, S.A., Miers, D.: BPMN Modeling and Reference Guide. Future Strategies Inc. (2008)
21. Kao, B., Garcia-Molina, H.: Deadline assignment in a distributed soft real-time system.
IEEE Transactions on Parallel and Distributed Systems 8(12) (1997) 1268–1274
22. Bussler, C.: Workflow instance scheduling with project management tools. In: DEXA ’98:
Proceedings of the 9th International Workshop on Database and Expert Systems Applica-
tions. (1998) 753
23. Son, J.H., Kim, J.H., Kim, M.H.: Deadline allocation in a time-constrained workflow. Int. J.
Cooperative Inf. Syst. 10(4) (2001) 509–530
24. Kochut, K.J., Sheth, A.P., Miller, J.A.: Orbwork: A corba-based fully distributed, scalable
and dynamic workflow enactment service for meteor. In: Department of Computer Science,
University of Georgia. (1999)