Information Quality (IQ) is particularly important for the efcient performance of any system. Despite this, most of the Requirements Engineering (RE) frameworks either ignore IQ needs, or they deal with them as mere technical issues, i.e., they do not consider the social and organizational aspects that underlie such needs. This paper summarizes the experience of the authors in modeling and analyzing IQ requirements for socio-technical systems. In particular, it summarizes the authors effort to propose an integrated RE framework that provides concepts for modeling and reasoning about IQ requirements since the early phases of the socio-technical system development.
Nowadays, Information Quality (IQ) is a growing concern for most organizations,
since they depend on information for managing their daily operations, taking
important decisions, etc., and relying on low quality information may negatively
in uence their overall performance, or it might even leads to disasters in the
case of critical systems (e.g., air tra c management systems, etc.). Despite this,
most existing Requirements Engineering (RE) frameworks and approaches either
loosely de ne, or simply ignore IQ requirements. Several techniques for dealing
with IQ have been proposed in the literature (e.g., integrity constraints).
However, most of them focus on the technical aspects, and do not solve problems that
may arise at organizational or social levels. More speci cally, these techniques do
not satisfy the needs of current complex systems, such as socio-technical systems
[
Fisher and Kingma [
In addition, most of these approaches provide ad-hoc techniques to deal with
IQ related vulnerabilities, instead of solving the main reason of such
vulnerabilities by considering them during the early phase of the system development (e.g.,
requirements level). In particular, a RE framework that enables for modeling
and reasoning about IQ requirements is still missed. To this end, we proposed a
novel RE framework that adopts the Goal-Oriented Requirements Engineering
(GORE) paradigm. Among the several GORE approaches o ered in the
literature (e.g., KAOS, i*, etc.), we adopted Secure Tropos [
Objectives of the research : as previously discussed, a RE framework for capturing IQ requirements is still missing. This paper summarizes our e ort to solve this problem by proposing a framework that is able to model and reason about IQ requirements. In particular, through this paper we summarize our work trying to answer the following research questions: RQ1: How can we analyze IQ in socio-technical systems? in other words, which IQ dimensions should be considered for analyzing IQ from sociotechnical perspective. We summaries our e ort to answer this question in section 3; RQ2: How can we model IQ requirements? we summaries our e ort to answer this question in section 4 by proposing concepts and constructs for modeling IQ requirements; RQ3: How can we verify the correctness of the IQ requirements model? we summaries our e ort to answer this question in section 5, by discussing a set of properties of the design that can be used to verify the correctness and consistency of the IQ requirements model.
RQ4: How can we support system designers in constructing the system? we summaries our e ort to answer this question in section 6 by proposing a methodological process to be followed by system designers during the di erent phases of the system design.
RQ5: How well does the framework perform when applied to realistic settings? we summaries our e ort to answer this question in section 7 by performing a set of experiments to verify whether the framework can e ciently perform the tasks it has been developed to do.
This experience report is structured as follows; Section (x2) describes our motivating example, while in Section (x3) we propose our multi-dimensional model for analyzing IQ. In Section (x4), we highlight the limitation in Secure Tropos for dealing with IQ requirements, and then we propose the required modeling concepts and constructs. Section (x5) presents the reasoning techniques that our framework o ers; and in Section (x6) we introduce the methodological process that underlies our framework. Section (x7) implements and evaluates the proposed framework. Finally, in Section (x8) we present our ongoing work, and we conclude the report at Section (x9). 2
Our motivating example concerns the May 6, 2010 Flash Crash, in which the Dow
Jones Industrial Average (DJIA) dropped about 1000 points (9% of its value).
Based on [
Stock markets are the places where traders gather and trade securities (e.g., NYSE, Chicago Mercantile Exchange (CME), NASDAQ). Markets have a main goal of making pro t by facilitating security trading. Usually, they manage traders' order matching and should ensure stable trading environment, which can be done by depending on their Circuit Breakers (CBs), where a CB is a technique that is used to slow down or halt trading to prevent a potential market crash. Furthermore, accounting rms are specialized for performing accounting activities, i.e., provide companies with clear and reliable information about their economic activities and the status of their assets. While auditing rms are specialized for providing e cient monitoring of the quality of information produced by companies concerning their nancial statements. Moreover, consulting rms are rms specialized for providing professional advices concerning nancial securities for a fee to traders and investors. Finally, credit assessment ratings rms are rms with a main objective of providing assessments of the credit worthiness of companies' securities, i.e., such rms help traders in deciding how risky it is to invest money in a certain security.
A deep analysis of the Flash Crash shows that several reasons that led to
the failure were caused by IQ related vulnerabilities. In what follows, we list
and discuss the main theories that have been proposed to explain the Crash:
(1) Fat- nger trade that is a human error caused by pressing a wrong key when
using a computer to input data; (2) The suspicious behavior of some HFTs that
negatively e ects the market prices and contributed to the Flash Crash [
IQ is a hierarchical multi-dimensional concept that can be characterized by di
erent dimensions/ sub-dimensions (e.g., accuracy, completeness, consistency, etc.
[
In [
Accessibility : the extent to which information is available, or easily and
quickly retrieved [
Believability : can be de ned as to which extent information is accepted or
regarded as true [
Trustworthiness : can be de ned as the extent to which information is credible
[
IQ
completeness
of the source
worthiness of the provision). Concerning our example, if markets apply a mechanism to analyze the trustworthiness of the trading orders, they will be able to detect \ ickering quotes" that have been provided by some HFTs, and apply the required mechanisms to mitigate their harmful e ect.
Accuracy : means that information should be true or error free with respect to
some known or measured value [
Completeness: means that all parts of information should be available, and
information should be complete for performing a task at hand [
Timeliness : means to which extent information is valid in term of time (e.g.,
su ciently up-to-date) [
T
Trade info T
[time] Stock market
Bank
NYSE
matchtriandgerasmong setnabvlireotnramdeinngt RfrbeoucmyeiovtRrr[eOad]dseeerlrsals/ndaofPpteeerrrfatotriaromdness enAvtrinraoadlniynPmzgeeanntdenvMtriraRa[odRn]nianmggeent
Order S list
NYSE part of info CME info Trade info information
[time] info provision
produce send modify Bank Manage order matching among traders and and
Ensure fair and stable trading environment
and Analyze the Manage
trading trading environment environment
CB info
Trade info
Legend
D goal/info D
goal delegation produceBy
decoamndp/oosrition neededBy
(a) Secure Tropos
Bank
DTT/ inf[otrimmaet]ion DTT/ ][RR []RO paiinnrtffooof [iPn]f[oRr]m[Mati]o[Sn] DTT/ [iPn]f[oRr]m[Mati]o[Sn] DTT/
truinstfo/dpisrtoruvsistsiofnor rerqeuairded oprteioadnal part of pdeerlmegiastsiioonn [dis]trduesltesgpaetiromnission
(b) Modeling extensions proposed in [
Consistency: means all multiple records of the same information should be the
same across time and space [
Secure Tropos (Figure 2 (a)) is able to capture the social and organizational
aspects of the system-to-be, but it does not o ers primitives for capturing IQ
requirements, i.e., it just deals with information whether they are available or
not and who is responsible about their provision. To tackle this problem, we
proposed a framework [
In particular, our modeling extensions can be classi ed under the following two sets: (I) Basic IQ concepts : that provide constructs for modeling seven IQ dimensions, and (II) Top-level IQ concepts : that are used to capture the IQ requirements of the stakeholders based on the actual information usage, and then gradually re ning them until reaching their operational speci cations. More speci cally, this set is used to identify how top-level IQ requirements can be captured and then re ned in terms of their di erent IQ dimensions, which can be modeled by the basic IQ concepts.
(I) Basic IQ concepts: in what follows, we summarize the concepts that
we have proposed in both [
Goal-Information relations : we re ned Secure Tropos goal-information relations by introducing four di erent concepts, (P)roduce indicates that an information item can be created by achieving the goal that is responsible of it producing; (R)ead indicates that a goal consumes an information item , and it can be either optional in which a goal can be achieved even if information was not consumed, or required in which information is required for the goal achievement; (M)odify : indicates that the goal achievement depends on modifying an information item; and (S)end : indicates that the goal achievement depends on transferring an information item to a speci c destination under prede ned criteria.
Accessibility : can be in uenced by the permissions that an actor has over information, which might enables or prevents it from using information as intended. Thus, we proposed four types of permissions concerning the four types of information usage that our framework supports (e.g., (P)roduce, (R)ead, (M)odify and (S)end). In addition, we extend the language to model permission delegation among actors, and to model trust/ distrusts concerning such permissions.
Believability : we extended read and produce concepts to accommodate believability check, since they are the only relations that can be in uenced by information believability.
Trustworthiness : is subject to (1) trustworthiness of the source that can be
captured by trust/distrust produce relations between information consumer and
its producer concerning the produced information; and the (2) trustworthiness
of the provision. In [
Accuracy : can be analyzed based on : (1) Accuracy of produced information that can be analyzed based on its believability, which enables to avoid producing unintended information, and its trustworthiness of the production process if the producing goal has been delegated; (2) Accuracy of provided information can be analyzed based on the trustworthiness of the provision; and (3) Accuracy of read information: can be analyzed based on information believability and its trustworthiness of the provenance.
Completeness: completeness can be subject to (1) value completeness for which we rely on Integrity Preserving provision (IP-provision) that preserves the integrity of the provided information; and (2) purpose of use completeness we rely on the \Part of" concept to model the relation between an information item and its sub-items.
Timeliness (validity): we extended information concept with a volatility
attribute to represent the change rate of information value [
Consistency : we extend the read relation between a goal and information with Purpose Of Use (POU) attribute that captures the intended purpose of information usage, which enables to identify interdependent readers that are actors who read the same information for the same POU. To this end, consistency can analyzed among interdependent readers based on their read times, i.e., information is consistent among its interdependent readers, if all of them have the same read time, otherwise it is inconsistent.
(II) High-level IQ concepts: in what follows, we summarize the concepts
that we have proposed in [
Top-level IQ softgoals : are softgoal concerning IQ requirements, and they are used as a starting point for identifying the stakeholders' needs concerning IQ.
And-decomposition for IQ softgoals re nement : since a softgoal can be re ned
into more speci c sub softgoals, if the joint satisfaction of these softgoals is
considered equivalent to satisfying the re ned softgoal [
Approximating leaf IQ Softgoals : we adopted the approximation relation
proposed by Jureta et al. [
Thus, to get better understanding of leaf IQ softgoals nature/type, and to
de ne the appropriate Information Quality Constraints (IQC)2 for their
approximation, we relied on Glinz [
In addition, for the approximation to be consistent with the di erent types of leaf IQ softgoals, we de ned three di erent types of IQCs: (1) Operational IQC: are constraints that de ne the required actions to be performed in already determined situations; (2) Declarative IQC: are constraints used to de ne properties of the system that should hold; and (3) Quantitative IQC: are constraints used to specify properties of the system that should hold, and can be measured on an ordinal scale. Table 1 shows how leaf IQ softgoals can be classi ed and approximated into the appropriate IQCs.
Finally, in order for the approximation relation between IQ softgoal and its
related IQC to hold, a well-de ned quality space should exist [
We used Disjunctive Datalog [
Pro1 states that the model should not include any top-level goal that is not achieved from the perspective of the actor, who aims for it. This property can be used to quickly verify the IQ requirements model, i.e., if this property holds for all top-level goals, we can infer that the requirements model is correct and consistent. Pro2 states that the model should not include any goal delegation chain, if there is no trust chain holds between the delegator and the delegatee, since delegation with no trust leaves the delegator with no guarantee about the achievement of its goal. Pro3 states that actors should have all information that is required for the achievement of the goals they are responsible of.
Pro4-6 are used to verify information permission related properties. For instance, Pro4 states that actors should have all the permissions they require to achieve their objectives. While Pro5 states that the model should not include actors who delegate permissions that they do not have. Pro6 states that the model should not include actors who have permissions, and there is no trust/ trust chain between such actors and information owner.
Pro7-9 are used to verify IQ related properties, i.e., the model should not include any information that does not ts for the purpose of use (e.g., produce, read, and send) from the perspectives of their user (e.g., producer (Pro7), reader (Pro8), and sender (Pro9)). Note that each of these properties covers several sub-properties. Consider Pro8 for example, information ts for read if it is accessible, accurate, complete, valid, and consistent.
Pro10-11 are used to ensure that the model manage separation of duties
among its actors to avoid any con ict of interest that may leads to di erent
kinds of vulnerability, i.e., such properties allow to identify situations at the
instance level that might endanger the system performance. In particular, they
state that the model should not include any agent that plays con icting roles in
terms of producing and/or reading information. The Enron scandal [
These properties allow for the automated analysis of the IQ requirements
model, i.e., the analysis relay on them to detect any design violation and notify
the designer about it. In other words, they enable for checking whether
stakeholders' IQ requirements are achieved or not, and identify the reason(s) preventing
their achievement (if any). In what follows, we list the main violations to the
properties of the design concerning the Flash Crash scenario that the automated
analysis was able to capture:
Inaccurate information: markets (e.g., CME ) consider information received
from any trader that plays the role of HFT trader as inaccurate, since no
trust in information production holds between them. In particular, HFTs
has the capability (trade large amount of securities in very fast rate) of
providing inaccurate information that is able to destabilize the trading
environment. According to [
Inaccurate information due to playing con icting roles: some companies are playing two con icting roles \accounting rm" and \auditing rm" concern producing \ nancial statement". Since we cannot trust a company for producing accurate \ nancial statement" concerning the companies they get paid from to perform their accounting services, such information is considered as inaccurate.
Unauthorized read due to playing con icting roles: some companies are playing two con icting roles \auditing rm" and \consulting rm" concern reading \security assessment", since they might use such information for providing paid consulting services. Thus, such companies should not has read permissions concerning \security assessment" information.
Incomplete information: \ NYSE CB info" and \NASDAQ CB info" that are used to stabilize the trading environment in NYSE and NASDAQ markets respectively, are identi ed as incomplete information from the perspectives of their readers, since they miss a sub part related to the purpose of use. However, this can be solved by providing NYSE and NASDAQ markets with the missed information sub items \CME CB info".
Inconsistent information: CME CB info is provided to NASDAQ and NYSE
with two di erent provision times 13 ms and 14.65 ms respectively [
Following Secure Tropos, our framework allows for a clear distinction between organizational (e.g., roles) and instance (e.g., agents) levels, and most of these properties apply to both of these levels. However, to avoid con icts between the organizational and instance levels that is a main issue in Secure Tropos, we are planning to model actors' interactions only at instance level. 6
Our framework [
IQ model Ex1- modeling IQ softgoals
Ex2- Approximating
IQ softgoals into IQC (4) Social interactions modeling : aims to identify and model the di erent social dependencies among actors of the system concerning information provision, goal and permission delegation. More speci cally, based on actors' capabilities some goals might be delegated to other actors, who have the capabilities to achieve them; and based on actors' needs, information/ permission is provided/ delegated to them; (5) Trust modeling : aims to model trust/ distrusts among actors concerning information producing, goal and permission delegation; (6) Analyzing IQ model : at this step the model is analyzed to verify whether all the stakeholders' requirements are achieved or not; and (7) Re ning IQ model : if some of the stockholders' requirements were not achieved during the model analysis step, the analyst try to nd solutions for such issues at this step.
In [
We evaluated the applicability and e ectiveness of our framework and its extended version depending on simulation method (experimental), i.e., execute artifact with arti cial data. To this end, we developed a prototype implementation of our framework (ST-IQ Tool)3 to test it for modeling and reasoning about IQ requirements. In what follows, we brie y describe our prototype, discuss its applicability and e ectiveness over the Flash Crash scenario, and then test the scalability of its reasoning support.
Implementation: our prototype consist of 4 main parts (ST-IQ Tool structure is shown in Figure 4): (1) Control component (JAVA based-program) that
controls and coordinates the three other components; (2) a graphical user interface (GUI) developed using Sirius4, which enables the system designers for drawing the IQ requirements model diagram by dragging-and-dropping modeling elements from palettes, and allows for specifying the properties of these elements along with their interrelations; (3) model-to-text transformation mechanism that supports the translating of the graphical requirements models into Disjunctive Datalog formal speci cations depending on Acceleo5; and nally (4) automated reasoning support (DLV system6) that takes the Disjunctive Datalog speci cation, which resulted from translating the graphical model along with the reasoning axioms, and then veri es the correctness and completeness of the requirements model against the properties of the design.
Applicability and e ectiveness: we evaluated our framework and its extended version by showing their applicability in capturing the IQ requirements along with its e ectiveness in identifying any violation to the properties of the design by applying it to the Flash Crash case study. In particular, we used our modeling language to model the Flash Crash, and then we translated the requirements diagram into Disjunctive Datalog formal language. Finally, we depend on the automated reasoning support to check whether the requirements model is correct and consistent, i.e., whether all the properties of the design hold. The analysis captured all violations of the properties of the design that we consider including inaccurate, incomplete, inconsistent information, etc.
Experiments on scalability: to test the scalability of the automated reasoning support, we investigated the reasoning execution time with respect to the model size. In particular, we proposed a model and then expanded it by increasing the number of its modeling elements through several steps. At each step, we performed an analysis test and we calculated the execution time. The result showed that the relation between the size of the model and the execution time is linear (not exponential). 8
We brie y discuss several ongoing works to improve our proposed framework.
Social trust analysis: all RE approaches including ours mainly focus on trust as social relations, without proposing any speci c modeling technique to relate them with the internal requirements of the system's components. In other words, existing approaches are able to model trust requirements, but o er no analysis mechanisms to verify their consistency with the di erent actors' competencies and motivations toward the trustum. We are currently working on extending our framework by proposing the required concepts and constructs for modeling and analyzing trust among actors of the systems based on sets of beliefs related to the actors' competencies (can do) and motivations (will do), which can be used to clearly identify \why" an actor can trust/ distrusts another one.
Automated speci cation of IQ policies: organizations rely on di erent
kinds of policies to de ne the permitted/ forbidden actors' activities toward
information in order to deal with IQ related concerns. Yet most of the proposed
solutions focus on the technical aspects of the system, which make them
inadequate for socio-technical systems. Thus, we extended our framework to support
the automatic derivation of the nal IQ policies from the IQ requirements model
[
Modeling and analyzing IQ requirements in Business Processes
(BP): driven by the needs of some system designers, who put more emphasis
on modeling BPs within the system rather than modeling the whole system, we
proposed an extension to our framework [
In this paper, we summarized our experience in modeling and analyzing IQ requirements for socio-technical systems. First, we argued that IQ is not only a technical issue, but it is also an organizational and social issue. Thus, any solution for IQ should consider the social and organizational context where the system will operate. Moreover, we highlighted the importance of capturing IQ requirements of the system from the early development phases.
Second, we showed how IQ can be analyzed through its di erent dimensions, and we proposed a multi-dimensional model for analyzing IQ from sociotechnical perspective. Third, we justify our choice in considering Secure Tropos as a baseline for our proposed framework, and we discussed its limitations for modeling and reasoning about IQ requirements. In addition, we discussed the required extensions of Secure Tropos for capturing IQ requirements, and then we extended its conceptual model with the required concepts. In particular, our proposed framework enables system designers to capture IQ requirements in terms of its di erent dimensions, and it provide the required analysis techniques to verify whether the IQ requirements are met or not. In addition, we discussed our methodological process that supports designers during the di erent phases of the system design.
We illustrated the utility of our framework by applying it to a stock market crash case study. More speci cally, we evaluated our framework by showing its ability in modeling and analyzing IQ requirements along with its e ectiveness in detecting any violations to the properties of the design. Moreover, we evaluated the scalability of its reasoning techniques by calculating the relation between the model sizes and the reasoning execution time, and the result proves that such relation is linear (not exponential). Finally, we discussed our ongoing research related to IQ requirements.
This research was partially supported by the ERC advanced grant 267856, \Lucretius: Foundations for Software Evolution", http://www.lucretius.eu/.