In this article, we propose to explore the role of quality evaluation to measure strategic alignment and support strategic IT requirements. In particular we focus on the eGovernment context that presents a set of characteristics asking for different evaluation metrics from the ones adopted for business in the private sector which are usually more investigated in terms of IT strategy alignment. Indeed, it is worth noting that, in the context of eGovernment, strategy is not oriented to obtain a competitive advantage and shareholder value as in the private sector; on the contrary, eGovernment value heavily depends on political and social objectives, such as trust in government, social inclusion, community regeneration, community well-being and sustainability [1]. In general terms, in the context of eGovernment, value relies mainly on pubic value. As stated by Grimsley and Meehan [1] on the basis of Moore's conceptualization [2], public value can be interpreted as the value that citizens and their representatives seek in relation to strategic outcomes and experiences of public services.
Due to these issues, we need to consider value not only from an economic perspective, but also as the degree to which public policies improve the quality of life of constituencies -namely citizens and businesses -by improving the quality of public services and of the public administration organization and processes. Therefore, a systemic perspective on quality is required. Among the available frameworks for eGovernment projects assessment [3], in this paper we consider GovQual [4] for its focus on quality dimensions and their mutual relationships for the different facets involved in eGovernment projects (i.e. economic, social, organizational, legal, and technological facets).
In this paper we consider a scenario focused on the Italian context, in which the chosen quality dimensions result from a preliminary analysis of online available eGovernment strategy documentations and we assume that they implement the political-visionrelated choice; while the scenario process is modelled on the basis of direct observation over a one-year period from September 2008 to September 2009. The corresponding quality evaluation exploits real-life data obtained from interviews and questionnaires filled by a first sample of non-EU foreign researchers and students in the scope of our Computer Science department, who have applied their residency permits within the past three years. Finally, a probabilistic simulation and analysis based on the sampled data have been carried out in order to elicit quality dimensions with priority among the ones considered.
The paper is organized as follows. Section 2 discusses related work. Section 3 describes the scenario for the application of the proposed approach, with a focus on the Italian context. Specifically, the considered administrative process is detailed in Section 3.1, while qualities considered in the scenario are modelled in Section 3.2. Finally, a prior computation for quality dimension evaluation is detailed in Section 3.3, based on which we present probabilistic analysis with Monte Carlo simulation in Section 3.4. Section 4 states conclusions and future works of the paper.
IT Strategy alignment has been considered as a major issue for management of information systems [5,6] in order to provide competitive advantages to IT enabled businesses. Strategic IT requirements engineering [7] aims to model alignment in order to elicit strategy related requirements, the most appropriate (among other facets) for the operational level of the considered domain of intervention [8]. To this end business modelling represents an emerging effort providing a set of frameworks for an explicit link between system requirements and the objectives of business strategy [9].
The most comprehensive and well-defined languages and frameworks for business modelling are the Resource-Event-Actor (REA) framework, the e3value framework, and the Business Model Ontology (BMO). The REA framework [10] is focused on representing increases and decreases of value in organizations, which has its origins in business accounting. The core concepts in REA are Resource, Event, and Actor, where every transaction can be described as two events in which two actors exchange resources. e3value framework [11] explicitly focuses on resources exchange as value objects. Basic concepts in the e3value ontology are actors, value objects, value ports, value interfaces, value activities and value exchanges. The BMO [12] provides a framework that consists of nine core concepts classified under four categories, as described in the following: the category product as a single concept, that is value proposition; the category customer interface has three concepts, namely target customer, distribution channel, and relationship; the category infrastructure management has three concepts, namely value configuration, capability, and partnership; the category financial aspect has two concepts, namely cost structure, revenue model.
Besides these frameworks which support a value-based documentation of the business model to be implemented in the systems, other models and approaches have been proposed in the literature, focusing more on strategies to model and measure the alignment as a source for requirement engineering, such as the Business Rules Group Motivation Model (BRG-Model) [13], the B-SCP framework [14], INSTAL method [15] (based on the Map Model [16] and exploiting quality metrics [17]), and the e3alignment model [18] as an extension of the e3value framework.
Considering now the operational level, i* [19] allows to model organizational goals and tasks together with actors and resources. Business process modelling is necessary as a representation of how the operational level works and is managed -in this paper we consider administrative processes as business processes -by means of the state of the art tools and concepts such as the Business Process Modelling Notation [20]. In particular, process models are the basis of the development of specific workflow applications. The models describe the process structure and logic on a type level, whereas workflow applications support the execution of single processes on the instance level.
In this paper we discuss a quality evaluation framework in order to explore its role in providing metrics which support the alignment of operational level with the strategy to be implemented in the eGovernment initiatives. Moreover, we focus on operational level representation by means of process models. Whereas is relevant that we discuss here the state of art models and frameworks for business modelling, this topic will be considered in future work.
In this section, we use the residency permit application process as a scenario to evaluate strategic alignment with respect to quality dimensions and to help analyzing relationships among them, in order to identify evolution requirements that lead to a better strategic alignment. The scenario is focused on the Italian context. According to the agreement signed by the Italian Ministry of Internal Affairs and Poste Italiane SPA (the major Italian company for postal services and mail), non-EU foreign students should present applications for residency permits in Italy annually. The residency permit for study reasons may only be applied and renewed if the entry visa was issued to attend a study course of more than one year in length. The first application, valid for one year, should be made within 8 days upon applicants' arrival in Italy and renewed annually. But in any case, it cannot be renewed for more than three years above the duration of the study course. In the following, we first introduce a workflow system, METEOR, to model the residency permit application process, and then two quality dimensions, efficiency and effectiveness, are mainly modelled w.r.t. different layers in GovQual framework; based on which we present quality computation methods and finally we use Monte Carlo simulation tools to analyze dependencies among quality dimensions.
Workflow Structure Existing workflow systems provide a set of indispensable functionalities that manage and streamline business process. Yet, few research groups have concentrated their efforts on enhancing workflow systems to support workflow Quality of Service management. Most of the research carried out in order to extend workflow systems' capabilities to include project management features has mainly been done for the time dimension [21][22][23]. In this paper, we adopt a workflow management system named METEOR [24] in which, not only is time considered, but also other dimensions associated with workflow executions, such as cost and efficiency. To compute quality of service dimensions, METEOR has developed both a model for the specification of workflow QoS and methods to analyze and monitor QoS, which can be exploited to evaluate different quality dimensions in the following scenario.
As defined in [24], a workflow is composed of tasks and transitions. Tasks are represented using a circle, networks (sub-workflows) using rounded rectangles, and transitions are represented using an arrow. Transitions express dependencies between tasks and are associated with an enabling probability (p 1 , p 2 , .., p n ). When a task has only one outgoing transition, the enabling probability is 1. In such a case, the probability can be omitted from the graph.
Residency Permit Workflow Description Residency Permit process is composed of four sub-processes (sub-workflow): Document Preparation, Application Identification, Fingerprint Collection, and Permit Release. Detailed process goes as follows (seen in Figure 1): -In the stage of Document Preparation, interested parties (non-EU foreign students) ask for related application kit available at certain post offices and fill the forms. Signed application forms together with other required documents must be sent by post to the Police Office (Questura) of their province of residence for the first issue of residency permit. A collection of documents include:
• Complete photocopy of passport or other equivalent document;
• Photocopy of the statement certifying the course of study to be taken, certified by the Italian Diplomatic/Consular mission when the entry visa is issued; • Photocopy of insurance policy, valid throughout the country and for the entire period of validity of the residency permit, against the risk of illness and injury; • Photocopy of the documentation certifying to the availability of adequate financial resources for the validity period of the residency permit. -In the stage of Application Identification, related immigration offices in Italy will check the completeness and correctness of received application documents, and publish the identification result on Internet, where applicants can query their application statements by a user ID and password assigned to the application receipt; -In the stage of Fingerprint Collection, once application documents are identified, in about three months or even longer applicants will receive a registered letter from the Immigration Office summoning them to be fingerprinted (the same information is sent to applicants by SMS as well). Meanwhile, each applicant can inquire his/her application statement physically to related immigration offices according to their application statements published on Internet. -In the stage of Permit Release, after the collection of fingerprint, applicants will have another appointment (the same information is sent to applicants by SMS as well) to release their regular residency permit. In other cases, they will get temporary permits valid for three months or no permit.
As shown in Figure 2 (text in bold), in this paper we mainly focus on the assessment of a limited set of quality dimensions: efficiency, which can be mutually influenced by other dimensions, such as effectiveness. Concerning efficiency, we distinguish between temporal, economic, and procedural efficiency. Temporal efficiency refers to efficient use of time in service production and provision. Temporal efficiency is significant in two layers. At the process/service layer it is classified in terms of two dimensions: the user time (the average time spent by users to obtain the service) and the service provision time (the average time spent by organizational units to produce the service). Temporal efficiency at the ICT infrastructure layer results in response time, the usual dimension considered for hardware and network infrastructures. The measure is simply the time span that the resources need to execute transactions and respond to queries. Economic efficiency concerns the costs sustained for service provision and their trends in time.
The procedural efficiency concerns the level of bureaucratic simplification of the administrative processes, meaningful for the organizational system layer. Procedural Fig. 2: Quality Dimensions in different layers of GovQual Framework efficiency refers to the obligations and constraints that laws impose on the administrative processes and on the interactions between administrations and citizens. We associate procedural efficiency with a metric consistent in the level of simplification. This is measured in terms of the number of interactions required by users to provide useful information in order to complete the service.
On the other hand, effectiveness is meaningful at the process/service layer and at the data layer. At the process/service layer it can be expressed in terms of the achievement of the users' expectations. At the data layer, effectiveness results in two dimensions: accuracy and completeness. In this paper we evaluate completeness in the data layer. We use both qualitative (user comments) and quantitative (number of complaints) indicators as metrics for effectiveness assessment in process/service layer. User complaints/comments reflect the users' perception of the accuracy and reliability of service provision, and the evaluation of the service-oriented attitude of personnel in charge of the service within organizational units. Table 1 shows a mapping from qualitative indicators (user comments) to quantitative indictors (number of complaints). Data com-
pleteness is the extent to which data is of sufficient breath, depth and scope for the task at hand. In most service execution processes, as in the residency permit scenario, the execution is incapable to proceed if data is not completely provided. In this sense we measure data completeness with a boolean < yes, no > domain. According to the above definitions of efficiency and effectiveness in different layers of GovQual framework, we can specify temporal and procedural efficiency, process/service and data effectiveness in residency permit application workflow as follows (Table 2): 1. Appointment Notification (1) refers to the first appointment for fingerprint collection and original document check; Appointment Notification (2) refers to the second notification for regular permit release. 2. Fingerprint Collection (1) refers to the fingerprint collection and original document check for regular permit processes, while Fingerprint Collection (2) refers to the fingerprint collection and original document check for temporary permit processes.
3. Since there are internal loop transactions within the task of Application Statement Inquiry (Figure 1), in Table 2 we only consider time spent for a single task of Application Statement Inquiry. 4. Probability distribution of each task is assigned on the basis of a few interviews.
Currently we consider two distributions, normal distribution N (µ, σ 2 ) and histogrm distribution Histogrm(min, max, {p 1 , p 2 , ..., p n }) (in this scenario, Appointment Notification (1) follows a distribution of Histogrm(2, 13, {0.5, 2.5, 2, 1})).
In the future stage, we plan to present more precise distributions by exploiting surveys over larger amount of application cases. In addition, due to the complexity of relationships among quality dimensions, for certain dimensions we will present their joint distributions instead of marginal distributions, i.e., joint distribution of process/service effectiveness and efficiency. 5. Level of Simplification of residency permit is dependent from other quality dimensions. Since interactions are fixed for other tasks in the workflow, the most flexible part is related to the task of Application Statement Inquiry. Considering that applicant/staff interactions for Application Statement Inquiry are subjective to applicants, we fix the frequency for inquiry in this scenario, which is also reasonable and practical in real life: suppose in the ideal case, applicants will get appointment notifications (both by post and by SMS) in 3 months after their applications have been identified. If not the case, applicants will wait for another 3 months and inquiry their application statements at related Immigration Offices physically for the first time, and once every 3 months if still necessary. Since in 12 months after the first application -which means maximally three interactions for application statement inquiry -applicants are obliged to renew their permits, the workflow will result in the following consequences: within one interaction, applicants may get their regular permits of stay; after two interactions, applications may get temporary permits; after three interactions applications may get no permits. Distributions of number of interactions and permit types will be analyzed in Section 3.4 where we observe a strong correlations between the two factors. 6. Effectiveness in Process/Service layer is presented by a probability distribution of user complaints (also considered as user comments) with respect to different results they get at the end of permit application workflow. Distribution of this quality dimension and the factors which have significant impacts on it are deduced from several interviews we made together with Monte Carlo simulations in Section 3.4. 7. Two simple observations can be achieved from Table 2. Different quality dimensions influence each other in two levels: (i) within single task among the same/different layer(s); (ii) over the whole service process among the same/different layer(s). A more statistic analysis of the above observations will be presented in Section 3.3.
In this section we compute overall time used for residency permit workflow, in order to evaluate relationships among different quality dimensions, i.e., efficiency and effectiveness, and to optimize quality requirements. Overall time for the whole workflow can be computed in three steps: (i) first we present formulas to calculate time spent for each atomic task; (ii) based on which we reduce complex task structures, including (1) sequential, (2) parallel, (3) conditional, (4) fault-tolerant, (5) loop, and ( 6) network, into atomic ones; in the following we show three complex structure reduction examples, for Document Collection, Application Kit Collection and Application Statement Inquiry, respectively; (iii) finally, according to the above modelings, we present the overall time spent for residency permit workflow in a more abstracted level.
Atomic Task Time Computation In Table 3, computation details for all tasks involved in residency permit application is presented, in which we assign an task ID, t i , to each task in the workflow. For each task t i in a service, the overall task time T (t i ) is computed as the sum of user time U T (t i ) and/or service provision time P T (t i ) and/or Response Time RT (t i ), all of which are considered as the main indictors to evaluate efficiency of a service. The overall task time is also effected by other quality dimensions in different layers. Relationships among them will be the focus of the next sections. 1. Since applicants will get Appointment Notification (2) for regular permit release at the end of their fingerprint collection task, we mainly consider the service provision time as time used for completing this task. Meanwhile, applicants will get the same reconfirmation information by SMS, which is measured by Response Time in ICT Infrastructure layer. However we consider such a process structure as a fault tolerant sub-workflow, and for the sake of simplicity, response time for Appointment Notification ( 2) is kept in Table 2 but omitted in the overall time calculation. 2. Fingerprint Collection (2) for temporary permit processes is executed at the end of Application Statement Inquiry task, thus we mainly consider service provision time as time spent for completing this task.
Complex Task Time Computation In the following we present complex structure reduction computations for three atomic tasks respectively: Document Collection, Application Kit Collection and Appointment Statement Inquiry (Figure 3). The loop structure be the reduced loop task of t 1 , then time spent for completing t l1 , denoted as T (t l1 ), is calculated as follows:
In which, f 1 (d) is the frequency of document preparation and is inversely proportional to data completeness:
Since data completeness for t 1 is uniformly distributed (see Table 2), its probability distribution P 1 is:
Similarly for Application Kit Preparation t 2 , of which we will not present details due to the reason of space.
As for Application Statement Inquiry t 6 , let t l6 be the reduced loop task of t 6 and random variable X as the number of interactions in fulfilling t 6 , in which {x = 0, 1, 2, 3} ∈ X. T (t l6 ), denoted as time spent for completing t l6 , the task is calculated as follows:
As similarly to t 1 and t 2 , t 6 influences different quality dimensions. However the impact is not only within single task but also involved a higher abstract level, in this scenario as Fingerprint Collection sub-workflow. Therefore we will focusing analyzing probability distributions in Fingerprint Collection sub-workflow in the following sections.
Sub-Workflow Time Computation Based on the above modelling, we abstract tasks involved in the residency permit workflow into four sub-workflows: Document Preparation, Application Identification, Fingerprint Collection, and Permit Release. We first compute time spent to fulfill each sub-workflow, and use the results for final overall time calculation. As shown in Figure 1, time spent for the sub-workflow of Document Preparation, denoted as T (sw 1 ), is computed as follows:
Time spent for the sub-workflow of Application Identification, denoted as T (sw 2 ), is computed as follows:
Before computing time spent for the sub-workflow of Fingerprint Collection, let's first recall that random variable X is defined as the number of interactions in fulfilling t 6 , in which {x = 0, 1, 2, 3} ∈ X. Similarly, we define random variable Y as permit types applicants will get at the end of residency permit application workflow, and random variable Z as user complaints / comments.
Time spent for Fingerprint Collection sub-workflow, denoted as T (sw 3 ), is computed as follows:
Quality Evaluation for Strategic Alignment Engineering 11
The time spent for the sub-workflow of Permit Release, denoted as T (sw 4 ), is computed as follows:
Finally, the overtime spent for completing the whole workflow of residency permit, denoted as T , is computed as follows:
In the next section, we will use the above quality modelling results as input for Monte Carlo simulation, in order to get a deeper and statistical analysis of quality evaluation for strategic alignment, and to further improve requirement re-engineering in business processes.
With the residency permit scenario modelled into the above workflow and sampled reallife data at hand, the idea of adopting Monte Carlo simulation for quality evaluation is inspired with no surprise. Specifically, the simulation iteratively runs over the workflow which imitates real-life residency permit application processes, where we can benefit from two aspects: (i) it overcomes our limited sample size and extends our observation with user-defined number of iterations; (ii) it enables statistic analysis and predictions over the underlying scenario which is complex, nonlinear and involves more than just a couple of uncertain parameters.
Based the above reasoning, we initialize the Monte Carlo simulation scenario (in Table 4) with our sampled real-life data as input parameters (in Table 2 and 3) and equations in Section 3.3 as uncertain functions. The expected outputs include a subset of efficiency and effectiveness indicators we will explicitly discuss in the following text. Simulation Results and Discussion Among several simulation statistics, the one of our interest is sensibility, i.e., analysis of which input factor having the most significant impact on the output. Sensibility of the model shows the relationships and dependencies of different quality dimensions of multiple layers in this underlying eGovernment application. Detailed sensibility statistics is available in Figure 4.
Focusing on temporal efficiency of the residency permit process, in sub-workflow 1 and 2, Data Completeness (Effectiveness in ICT layer) and Service Provision Time have strong influence on the sub-workflow time. However focusing on the total time spent for the overall workflow, we discover User Time (in this scenario, user time of Appointment Notification(1)) affecting the overall temporal efficiency.
On the other hand, the level of simplification of the considered scenario depends on the number of inquiries applicants made for application statement check, which is affected by the time applicants wait for their first appointment notifications. The analysis points out that User Time of Appointment Notification (1) has a strong impact on level of simplification (with correlation coefficient value of 0.949 in the range of [−1...1], meaning that the longer the user time for appointment notification (1) is, the more interactions the process has). Since level of simplification is a joint distribution of X (the number of interactions) and Y (permit type), we also simulate the sensibility factor of the latter (permit type). Resulted data shows that the type of permit an applicant gets at the end of the process is also affected by user time of the first appointment notification (with correlation coefficient value of −0.818, meaning the longer the user time for appointment notification (1), the permit type resulted from the process can vary from regular, temporary to no permit released). The above analysis indicates a strong dependency between X and Y .
Furthermore, sensibility analysis of effectiveness in process/service layer (in the form of user comments, denoted as Z) is an interesting observation. We categorize Z according to the overall time spent for residency permit for the following reasons: (i) overall time spent directly covers temporal efficiency in different layers; (ii) it is also affected by data completeness; (iii) different types of permit result in different overall time spent, which in turn is affected by level of simplification in organization system layer. In another words, the output of overall workflow time can be considered as a representation of all input quality dimensions in the scenario, and therefore it is reasonable to assume that sensibility analysis of effectiveness can be observed from the correlation between overall workflow time and Z. Statistic result in Figure 4 shows that, among all possible factors, process/service effectiveness is again heavily influenced by user time (with correlation coefficient value of −0.712, meaning the longer applicants wait, the less effectively they could benefit from the underlying service).
In this article, we have explored the role of quality metrics to measure strategic alignment and support strategic IT requirements. In particular we have focused on the eGovernment context. To this end we have adopted a framework for systemic quality assessment, considering qualities for the different facets of the subject domain (process/service, organizational system, and ICT technologies). Values for different qual-Fig. 4: Sensibility Statistics of Residency Permit Scenario ities have been defined on the basis of results from interviews. We have then applied a probabilistic model to identify relationships and dependencies between the quality dimensions. In particular, we have identified that i) user time of appointment notification has a strong impact on level of simplification and consequently ii) type of permit an applicant gets at the end of the process and the degree of client satisfaction of the service scenario are also affected by user time of the first appointment notification. The obtained results indicate that, from an alignment perspective, the probabilistic analysis together with the quality framework have provided useful information about critical factors at operational level for improving services related to residency permit, namely the relevance of user time with respect to the type of permit required. Yet actual government strategy focuses on the improvement of service provision time at the back-end level.
Limitations of the research are related to (i) the limited set of qualities considered, (ii) the relative small sample of interviews and questionnaires collected, and (iii) the lack of a formal model to map intentions at strategy level with modelling of workflow at operational level. In further works we will exploit a survey approach to extend the sample size of interviewees for analysis and extend the actual framework by adopting a formal model for strategy representation.