This article explores the role of quality evaluation to measure strategic alignment and support strategic IT requirements. In particular, the focus is on eGovernment context that presents a set of characteristics asking for different evaluation metrics from the ones adopted for business in the private sector (usually more investigated in terms of IT strategy alignment). The proposal approach implements a subset of a quality framework on a real scenario for the purpose of quality evaluation. It exploits real-life data from interviews and questionnaires filled by a first sample of non-EU foreign researchers and students in the scope of the Computer Science department where the authors are affiliated, who have applied their residency permits within the past three years. Finally, a probabilitybased analysis is carried out in order to elicit quality dimensions with priority among the ones considered.
In this article, we propose to explore the role of quality evaluation to measure strategic
alignment and support strategic IT requirements. In particular we focus on the
eGovernment context that presents a set of characteristics asking for different evaluation
metrics from the ones adopted for business in the private sector which are usually more
investigated in terms of IT strategy alignment. Indeed, it is worth noting that, in the
context of eGovernment, strategy is not oriented to obtain a competitive advantage and
shareholder value as in the private sector; on the contrary, eGovernment value heavily
depends on political and social objectives, such as trust in government, social
inclusion, community regeneration, community well-being and sustainability [
Due to these issues, we need to consider value not only from an economic
perspective, but also as the degree to which public policies improve the quality of life
of constituencies - namely citizens and businesses - by improving the quality of
public services and of the public administration organization and processes. Therefore, a
systemic perspective on quality is required. Among the available frameworks for
eGovernment projects assessment [
In this paper we consider a scenario focused on the Italian context, in which the chosen quality dimensions result from a preliminary analysis of online available eGovernment strategy documentations and we assume that they implement the political-visionrelated choice; while the scenario process is modelled on the basis of direct observation over a one-year period from September 2008 to September 2009. The corresponding quality evaluation exploits real-life data obtained from interviews and questionnaires filled by a first sample of non-EU foreign researchers and students in the scope of our Computer Science department, who have applied their residency permits within the past three years. Finally, a probabilistic simulation and analysis based on the sampled data have been carried out in order to elicit quality dimensions with priority among the ones considered.
The paper is organized as follows. Section 2 discusses related work. Section 3 describes the scenario for the application of the proposed approach, with a focus on the Italian context. Specifically, the considered administrative process is detailed in Section 3.1, while qualities considered in the scenario are modelled in Section 3.2. Finally, a prior computation for quality dimension evaluation is detailed in Section 3.3, based on which we present probabilistic analysis with Monte Carlo simulation in Section 3.4. Section 4 states conclusions and future works of the paper. 2
IT Strategy alignment has been considered as a major issue for management of
information systems [
The most comprehensive and well-defined languages and frameworks for business
modelling are the Resource-Event-Actor (REA) framework, the e3value framework, and
the Business Model Ontology (BMO). The REA framework [
Besides these frameworks which support a value-based documentation of the
business model to be implemented in the systems, other models and approaches have been
proposed in the literature, focusing more on strategies to model and measure the
alignment as a source for requirement engineering, such as the Business Rules Group
Motivation Model (BRG-Model) [
Considering now the operational level, i* [
In this paper we discuss a quality evaluation framework in order to explore its role in providing metrics which support the alignment of operational level with the strategy to be implemented in the eGovernment initiatives. Moreover, we focus on operational level representation by means of process models. Whereas is relevant that we discuss here the state of art models and frameworks for business modelling, this topic will be considered in future work. 3
In this section, we use the residency permit application process as a scenario to evaluate strategic alignment with respect to quality dimensions and to help analyzing relationships among them, in order to identify evolution requirements that lead to a better strategic alignment. The scenario is focused on the Italian context. According to the agreement signed by the Italian Ministry of Internal Affairs and Poste Italiane SPA (the major Italian company for postal services and mail), non-EU foreign students should present applications for residency permits in Italy annually. The residency permit for study reasons may only be applied and renewed if the entry visa was issued to attend a study course of more than one year in length. The first application, valid for one year, should be made within 8 days upon applicants’ arrival in Italy and renewed annually. But in any case, it cannot be renewed for more than three years above the duration of the study course. In the following, we first introduce a workflow system, METEOR, to model the residency permit application process, and then two quality dimensions, efficiency and effectiveness, are mainly modelled w.r.t. different layers in GovQual framework; based on which we present quality computation methods and finally we use Monte Carlo simulation tools to analyze dependencies among quality dimensions. 3.1
Workflow Structure Existing workflow systems provide a set of indispensable
functionalities that manage and streamline business process. Yet, few research groups have
concentrated their efforts on enhancing workflow systems to support workflow Quality
of Service management. Most of the research carried out in order to extend workflow
systems’ capabilities to include project management features has mainly been done for
the time dimension [
As defined in [
Residency Permit Workflow Description Residency Permit process is composed of four sub-processes (sub-workflow): Document Preparation, Application Identification, Fingerprint Collection, and Permit Release. Detailed process goes as follows (seen in Figure 1): – In the stage of Document Preparation, interested parties (non-EU foreign students) ask for related application kit available at certain post offices and fill the forms. Signed application forms together with other required documents must be sent by post to the Police Office (Questura) of their province of residence for the first issue of residency permit. A collection of documents include: ² Complete photocopy of passport or other equivalent document; ² Photocopy of the statement certifying the course of study to be taken, certified by the Italian Diplomatic/Consular mission when the entry visa is issued; ² Photocopy of insurance policy, valid throughout the country and for the entire period of validity of the residency permit, against the risk of illness and injury; ² Photocopy of the documentation certifying to the availability of adequate financial resources for the validity period of the residency permit. – In the stage of Application Identification, related immigration offices in Italy will check the completeness and correctness of received application documents, and publish the identification result on Internet, where applicants can query their application statements by a user ID and password assigned to the application receipt; – In the stage of Fingerprint Collection, once application documents are identified, in about three months or even longer applicants will receive a registered letter from the Immigration Office summoning them to be fingerprinted (the same information is sent to applicants by SMS as well). Meanwhile, each applicant can inquire his/her application statement physically to related immigration offices according to their application statements published on Internet. – In the stage of Permit Release, after the collection of fingerprint, applicants will have another appointment (the same information is sent to applicants by SMS as well) to release their regular residency permit. In other cases, they will get temporary permits valid for three months or no permit. 3.2
As shown in Figure 2 (text in bold), in this paper we mainly focus on the assessment of a limited set of quality dimensions: efficiency, which can be mutually influenced by other dimensions, such as effectiveness. Concerning efficiency, we distinguish between temporal, economic, and procedural efficiency. Temporal efficiency refers to efficient use of time in service production and provision. Temporal efficiency is significant in two layers. At the process/service layer it is classified in terms of two dimensions: the user time (the average time spent by users to obtain the service) and the service provision time (the average time spent by organizational units to produce the service). Temporal efficiency at the ICT infrastructure layer results in response time, the usual dimension considered for hardware and network infrastructures. The measure is simply the time span that the resources need to execute transactions and respond to queries. Economic efficiency concerns the costs sustained for service provision and their trends in time.
The procedural efficiency concerns the level of bureaucratic simplification of the administrative processes, meaningful for the organizational system layer. Procedural efficiency refers to the obligations and constraints that laws impose on the administrative processes and on the interactions between administrations and citizens. We associate procedural efficiency with a metric consistent in the level of simplification. This is measured in terms of the number of interactions required by users to provide useful information in order to complete the service.
On the other hand, effectiveness is meaningful at the process/service layer and at the data layer. At the process/service layer it can be expressed in terms of the achievement of the users’ expectations. At the data layer, effectiveness results in two dimensions: accuracy and completeness. In this paper we evaluate completeness in the data layer. We use both qualitative (user comments) and quantitative (number of complaints) indicators as metrics for effectiveness assessment in process/service layer. User complaints/comments reflect the users’ perception of the accuracy and reliability of service provision, and the evaluation of the service-oriented attitude of personnel in charge of the service within organizational units. Table 1 shows a mapping from qualitative indicators (user comments) to quantitative indictors (number of complaints). Data completeness is the extent to which data is of sufficient breath, depth and scope for the task at hand. In most service execution processes, as in the residency permit scenario, the execution is incapable to proceed if data is not completely provided. In this sense we measure data completeness with a boolean < yes; no > domain. According to the above definitions of efficiency and effectiveness in different layers of GovQual framework, we can specify temporal and procedural efficiency, process/service and data effectiveness in residency permit application workflow as follows (Table 2): 1. Appointment Notification (1) refers to the first appointment for fingerprint collection and original document check; Appointment Notification (2) refers to the second notification for regular permit release. 2. Fingerprint Collection (1) refers to the fingerprint collection and original document check for regular permit processes, while Fingerprint Collection (2) refers to the fingerprint collection and original document check for temporary permit processes. 3. Since there are internal loop transactions within the task of Application Statement Inquiry (Figure 1), in Table 2 we only consider time spent for a single task of Application Statement Inquiry. 4. Probability distribution of each task is assigned on the basis of a few interviews.
Currently we consider two distributions, normal distribution N (¹; ¾2) and histogrm distribution Histogrm(min; max; fp1; p2; :::; png) (in this scenario, Appointment Notification (1) follows a distribution of H istogrm(2; 13; f0:5; 2:5; 2; 1g)). In the future stage, we plan to present more precise distributions by exploiting surveys over larger amount of application cases. In addition, due to the complexity of relationships among quality dimensions, for certain dimensions we will present their joint distributions instead of marginal distributions, i.e., joint distribution of process/service effectiveness and efficiency. 5. Level of Simplification of residency permit is dependent from other quality dimensions. Since interactions are fixed for other tasks in the workflow, the most flexible part is related to the task of Application Statement Inquiry. Considering that applicant/staff interactions for Application Statement Inquiry are subjective to applicants, we fix the frequency for inquiry in this scenario, which is also reasonable and practical in real life: suppose in the ideal case, applicants will get appointment notifications (both by post and by SMS) in 3 months after their applications have been identified. If not the case, applicants will wait for another 3 months and inquiry their application statements at related Immigration Offices physically for the first time, and once every 3 months if still necessary. Since in 12 months after the first application - which means maximally three interactions for application statement inquiry - applicants are obliged to renew their permits, the workflow will result in the following consequences: within one interaction, applicants may get their regular permits of stay; after two interactions, applications may get temporary permits; after three interactions applications may get no permits. Distributions of number of interactions and permit types will be analyzed in Section 3.4 where we observe a strong correlations between the two factors. 6. Effectiveness in Process/Service layer is presented by a probability distribution of user complaints (also considered as user comments) with respect to different results they get at the end of permit application workflow. Distribution of this quality dimension and the factors which have significant impacts on it are deduced from several interviews we made together with Monte Carlo simulations in Section 3.4. 7. Two simple observations can be achieved from Table 2. Different quality dimensions influence each other in two levels: (i) within single task among the same/different layer(s); (ii) over the whole service process among the same/different layer(s). A more statistic analysis of the above observations will be presented in Section 3.3. 3.3
In this section we compute overall time used for residency permit workflow, in order to evaluate relationships among different quality dimensions, i.e., efficiency and effectiveness, and to optimize quality requirements. Overall time for the whole workflow can be computed in three steps: (i) first we present formulas to calculate time spent for each atomic task; (ii) based on which we reduce complex task structures, including (1) sequential, (2) parallel, (3) conditional, (4) fault-tolerant, (5) loop, and (6) network, into atomic ones; in the following we show three complex structure reduction examples, for Document Collection, Application Kit Collection and Application Statement Inquiry, respectively; (iii) finally, according to the above modelings, we present the overall time spent for residency permit workflow in a more abstracted level.
Atomic Task Time Computation In Table 3, computation details for all tasks involved in residency permit application is presented, in which we assign an task ID, ti, to each task in the workflow. For each task ti in a service, the overall task time T (ti) is computed as the sum of user time U T (ti) and/or service provision time P T (ti) and/or Response Time RT (ti), all of which are considered as the main indictors to evaluate efficiency of a service. The overall task time is also effected by other quality dimensions in different layers. Relationships among them will be the focus of the next sections. 1. Since applicants will get Appointment Notification (2) for regular permit release at the end of their fingerprint collection task, we mainly consider the service provision time as time used for completing this task. Meanwhile, applicants will get the same reconfirmation information by SMS, which is measured by Response Time in ICT Infrastructure layer. However we consider such a process structure as a fault tolerant sub-workflow, and for the sake of simplicity, response time for Appointment Notification (2) is kept in Table 2 but omitted in the overall time calculation. 2. Fingerprint Collection (2) for temporary permit processes is executed at the end of Application Statement Inquiry task, thus we mainly consider service provision time as time spent for completing this task.
Complex Task Time Computation In the following we present complex structure reduction computations for three atomic tasks respectively: Document Collection, Application Kit Collection and Appointment Statement Inquiry (Figure 3). The loop structure (a) Document Collection
(b) Appl. Kit Collection (c) App. Statement Inquiry
Fig. 3: Complex Task Time Computation of t1 Document Preparation is influenced by data completeness in Data layer. Let tl1 be the reduced loop task of t1, then time spent for completing tl1, denoted as T (tl1), is calculated as follows:
T (tl1) = T (t1)(1 + f1(d)) In which, f1(d) is the frequency of document preparation and is inversely proportional to data completeness: f1(d) = (0 1
DataCompleteness := Y es
DataCompleteness := N o Since data completeness for t1 is uniformly distributed (see Table 2), its probability distribution P1 is:
P1(d) = (0:5 Similarly for Application Kit Preparation t2, of which we will not present details due to the reason of space.
As for Application Statement Inquiry t6, let tl6 be the reduced loop task of t6 and random variable X as the number of interactions in fulfilling t6, in which fx = 0; 1; 2; 3g 2 X. T (tl6), denoted as time spent for completing tl6, the task is calculated as follows:
T (tl6) = xT (t6) As similarly to t1 and t2, t6 influences different quality dimensions. However the impact is not only within single task but also involved a higher abstract level, in this scenario as Fingerprint Collection sub-workflow. Therefore we will focusing analyzing probability distributions in Fingerprint Collection sub-workflow in the following sections. (4) (5) (6) (7) (8) (9) Time spent for the sub-workflow of Application Identification, denoted as T (sw2), is computed as follows:
T (sw2) = T (t4) + T (t5) Before computing time spent for the sub-workflow of Fingerprint Collection, let’s first recall that random variable X is defined as the number of interactions in fulfilling t6, in which fx = 0; 1; 2; 3g 2 X. Similarly, we define random variable Y as permit types applicants will get at the end of residency permit application workflow, and random variable Z as user complaints / comments.
Sub-Workflow Time Computation Based on the above modelling, we abstract tasks involved in the residency permit workflow into four sub-workflows: Document Preparation, Application Identification, Fingerprint Collection, and Permit Release. We first compute time spent to fulfill each sub-workflow, and use the results for final overall time calculation.
As shown in Figure 1, time spent for the sub-workflow of Document Preparation, denoted as T (sw1), is computed as follows:
T (sw1) = maxfT (tl1); T (tl2)g + T (t3)
RegularP ermit T emporaryP ermit N oP ermit
P erf ect Good Satisf actory P oor
U nacceptable 81 > Y = <0 > :¡1 >82 > >>>1 > Z = <0 > >>>¡1 > > :¡2 8T (t7) + T (t9) > < >:T (tl6) Time spent for Fingerprint Collection sub-workflow, denoted as T (sw3), is computed as follows:
T (sw3) =
T (tl6) + T (t8) + T (t10) fy = 1g fy = 0g fy = ¡1g The time spent for the sub-workflow of Permit Release, denoted as T (sw4), is computed as follows:
T (sw4) =
T (t11) 8T (t12) + T (t13) y = 1 > < >:0 y = 0 y = ¡1 Finally, the overtime spent for completing the whole workflow of residency permit, denoted as T , is computed as follows:
T = T (sw1) + T (sw2) + T (sw3) + T (sw4) (10) (11) In the next section, we will use the above quality modelling results as input for Monte Carlo simulation, in order to get a deeper and statistical analysis of quality evaluation for strategic alignment, and to further improve requirement re-engineering in business processes. 3.4
With the residency permit scenario modelled into the above workflow and sampled reallife data at hand, the idea of adopting Monte Carlo simulation for quality evaluation is inspired with no surprise. Specifically, the simulation iteratively runs over the workflow which imitates real-life residency permit application processes, where we can benefit from two aspects: (i) it overcomes our limited sample size and extends our observation with user-defined number of iterations; (ii) it enables statistic analysis and predictions over the underlying scenario which is complex, nonlinear and involves more than just a couple of uncertain parameters.
Based the above reasoning, we initialize the Monte Carlo simulation scenario (in Table 4) with our sampled real-life data as input parameters (in Table 2 and 3) and equations in Section 3.3 as uncertain functions. The expected outputs include a subset of efficiency and effectiveness indicators we will explicitly discuss in the following text. Simulation Results and Discussion Among several simulation statistics, the one of our interest is sensibility, i.e., analysis of which input factor having the most significant impact on the output. Sensibility of the model shows the relationships and dependencies of different quality dimensions of multiple layers in this underlying eGovernment application. Detailed sensibility statistics is available in Figure 4.
Focusing on temporal efficiency of the residency permit process, in sub-workflow 1 and 2, Data Completeness (Effectiveness in ICT layer) and Service Provision Time have strong influence on the sub-workflow time. However focusing on the total time spent for the overall workflow, we discover User Time (in this scenario, user time of Appointment Notification(1)) affecting the overall temporal efficiency.
On the other hand, the level of simplification of the considered scenario depends on the number of inquiries applicants made for application statement check, which is affected by the time applicants wait for their first appointment notifications. The analysis points out that User Time of Appointment Notification (1) has a strong impact on level of simplification (with correlation coefficient value of 0:949 in the range of [¡1:::1], meaning that the longer the user time for appointment notification (1) is, the more interactions the process has). Since level of simplification is a joint distribution of X (the number of interactions) and Y (permit type), we also simulate the sensibility factor of the latter (permit type). Resulted data shows that the type of permit an applicant gets at the end of the process is also affected by user time of the first appointment notification (with correlation coefficient value of ¡0:818, meaning the longer the user time for appointment notification (1), the permit type resulted from the process can vary from regular, temporary to no permit released). The above analysis indicates a strong dependency between X and Y .
Furthermore, sensibility analysis of effectiveness in process/service layer (in the form of user comments, denoted as Z) is an interesting observation. We categorize Z according to the overall time spent for residency permit for the following reasons: (i) overall time spent directly covers temporal efficiency in different layers; (ii) it is also affected by data completeness; (iii) different types of permit result in different overall time spent, which in turn is affected by level of simplification in organization system layer. In another words, the output of overall workflow time can be considered as a representation of all input quality dimensions in the scenario, and therefore it is reasonable to assume that sensibility analysis of effectiveness can be observed from the correlation between overall workflow time and Z. Statistic result in Figure 4 shows that, among all possible factors, process/service effectiveness is again heavily influenced by user time (with correlation coefficient value of ¡0:712, meaning the longer applicants wait, the less effectively they could benefit from the underlying service). 4
In this article, we have explored the role of quality metrics to measure strategic alignment and support strategic IT requirements. In particular we have focused on the eGovernment context. To this end we have adopted a framework for systemic quality assessment, considering qualities for the different facets of the subject domain (process/service, organizational system, and ICT technologies). Values for different qual
Fig. 4: Sensibility Statistics of Residency Permit Scenario ities have been defined on the basis of results from interviews. We have then applied a probabilistic model to identify relationships and dependencies between the quality dimensions. In particular, we have identified that i) user time of appointment notification has a strong impact on level of simplification and consequently ii) type of permit an applicant gets at the end of the process and the degree of client satisfaction of the service scenario are also affected by user time of the first appointment notification. The obtained results indicate that, from an alignment perspective, the probabilistic analysis together with the quality framework have provided useful information about critical factors at operational level for improving services related to residency permit, namely the relevance of user time with respect to the type of permit required. Yet actual government strategy focuses on the improvement of service provision time at the back-end level.
Limitations of the research are related to (i) the limited set of qualities considered, (ii) the relative small sample of interviews and questionnaires collected, and (iii) the lack of a formal model to map intentions at strategy level with modelling of workflow at operational level. In further works we will exploit a survey approach to extend the sample size of interviewees for analysis and extend the actual framework by adopting a formal model for strategy representation.