Software producing organizations regularly make complex technology decisions as part of the software production process, even when there is low availability of expertise in the domain within the organization. Software production is consequently a suitable domain to deploy decision support systems that intelligently assist decision-makers in selecting desirable technologies according to their requirements and priorities. In this tool paper, we introduce a decision support system that supports decision-makers in group and individual decision-making problems to select the most suitable technologies. Technology experts from di erent domains, such as database management systems and cloud service providers, con rm that the approach provides more knowledge than they could have collected independently, increases insight into the selection process, and reduces the time and cost of the decision-making process.
Thanks to rapidly evolving technologies and an abundance of technology vendors with homogeneous o erings in the marketplace, selecting suitable technologies for software producing organizations is a challenging process. Software architects and senior developers, who are not experts in each decision domain, need trusted partners who can e ectively understand their requirements and priorities as well as have deep technical expertise in software technologies in the market. Technology selection is the process of assessing the potential value of technologies and their contribution to the competitiveness and pro tability of Software Producing Organizations. The selection process is complicated because there are many factors, such as suitability and cost, that have to be considered. The technology
This work is a result of the AMUSE project. See amuse-project.org for more information.
Copyright 2018 for this paper by its authors. Copying permitted for private and academic purposes. selection process can be modeled as a multi-criteria decision-making (MCDM) problem that deals with the assessing of a set of alternatives, and considering a set of decision criteria.
This study introduces a Decision Support System Tool (DSST) that helps decision-makers with MCDM problems, such as Database Management System and Cloud Service Provider (CSP) selection problems. 2
In recent years researchers introduced a considerable variety of techniques, methods, and tools to solve di erent technology selection problems for Software Producing Organizations. Many variations exist, but all share the crucial phases of the decision-making process. The majority of the techniques in the literature use pairwise comparison as the main method to assess the weight of criteria. The pairwise comparison is a time-consuming process, and gets more complicated as the number of criteria increases. Some of the methods, such as Analytic Hierarchy Process, are not scalable, so in the case of modifying the list of alternatives or criteria, the whole process of evaluation should be conducted repeatedly. Therefore, these methods are costly and applicable for only a small number of criteria and alternatives. The majority of the MCMD techniques in literature de ne domain-speci c quality attributes to evaluate the alternatives. Such studies are mainly appropriate for speci c case studies. Furthermore, the results of these MCDM approaches are valid for a speci ed period, so by technology advances and new service o ering they will be out-of-date.
The Commercial-O -The-Shelf (COTS) selection problem is a subclass of
MCDM problems. Numerous approaches have been proposed for the general
problem of software component evaluation and selection. Becker et al. [
The DSST is unique in that it utilizes standard concepts from software
production: the MoSCoW prioritization technique (MoSCoW ) [
The fundamental components of a typical DSS [
The DataBase management system is a set of domain features facts related to the MCDM problem. Furthermore, it contains previous solutions, best practices, and lesson learned from prior made decisions. The Model-Base management system is a collection of rules, heuristics, and knowledge related to the MCDM problem. The Dialog Generation management system is a user interface which is used to acquire decision-makers' requirements and preferences.
The Inference Engine (IE) of the decision support system infers solutions. The IE receives domain feature requirements and their priorities according to MoSCoW from the Dialog Generation management system as its input. Next, it nds the most relevant rules from a collection of models in the Model-Base management system. Then, the IE by using facts from the DataBase management system deduces decisions. Eventually, it sends ranked feasible solutions to the Dialog Generation management system. The IE is a protocol for navigating through the rules and facts in a DSS to solve the problem. The IE infers solutions of MCDM problems based on the interaction of domain features facts and rules. In a standard DSS, the IE does not rely on knowledge base facts and rules, so it works independently from the other components.
The DSST3 comprises all of the fundamental components of a standard DSS. Figure 1 shows the fundamental components of the DSST and their relationships.
Dialog Generation management
system Requirements & Priorities (MoSCoW) Ranked Feasible Solutions
Inference Engine
Knowledge base Model-Base management system
Database management system Domain Qualities Features Alternatives Decision Meta-Model
CSP Decision Model
Domain Feature Facts
Previous Solutions Best Practices
Lessons Learned
DBMS Decision Model 3 We have invested roughly 960 hours into the implementation of the online Decision Model Studio (http://dss.amuse-project.org) to build decision models for MCDM problems in Software Producing Organizations. 3.1
The DSST is a culmination of several research projects combined. The DSST is central in four research projects with the DSST: one concerning DBMSs, one concerning CSPs, one concerning blockchain platforms, and one concerning Software Architecture decisions. For each of the projects, several experts were approached to evaluate the quality model, the COTS features, and the e ectiveness of the tool. In total, twenty-three experts (three DSS experts, two academics, ve Software Developers, six cloud consultants, three cloud architects, and four software architects) participated in this research to evaluate the e ectiveness and e ciency of the DSST. We selected the experts according to their expertise and experience that they mentioned in their professional pro le. 3.2
A decision model in the knowledge base of the DSST contains all facts and rules of an MCDM problem. In other words, a decision model de nes a decision graph to solve a speci c MCDM problem. The Decision Meta-Model is the base structure of a decision model in the knowledge base. It includes two primary sets (Qualities and Features ). The set Qualities is a set that keeps software quality attributes, and the set Features is a set that retains domain features of an MCDM problem.
The DSST uses the ISO/IEC 25010 standard [
Technology alternatives of an MCDM problem and the mapping between
them and domain features are determined based on documentation of
alternatives, literature studies, social networks, alternative experts, etc. As mentioned
earlier, the DSST utilizes MoSCoW to de ne decision-makers' domain feature
requirements and assess the importance of required domain features. Suppose
WMoSCoW = fwMust; wShould; wCould; wW on0tg is the set of priority weights
according to the de nition of the MoSCoW [
Each decision model de nes a decision structure for an MCDM problem systematically. The Knowledge Base is a collection of decision models, which are groups of rules and facts. The Inf erenceEngine ranks the alternatives based on their calculated scores. The score calculation process begins with computing the weight of each aspect of the decision model.The score calculation process is based on the well-known Weighted Sum Model. The scores of feasible solutions are more than zero. Moreover, by sorting the feasible solutions in descending order of their scores, the nal ranked feasible solutions will be given as the result of the DSST.Figure 3 and Figure 4 demonstrate a part of the decision graph and top-10 solutions for the Software Producing Organization respectively. 4 AFAS Software is an ERP vendor in the Netherlands with approximately 350 employees. One of AFAS' current challenges is validating whether they have chosen the right CSP for the new version of their product. The DSST provides a discussion and negotiation platform to enable Software Producing Organizations to make group decisions. It detects and highlights the con icts in the assigned priorities to the domain feature requirements by decision-makers, asks them to resolve disagreements. The DSST asks decisionmakers to de ne individual domain feature requirements based on MoSCoW. Next, it collects the individual prioritized domain feature requirements of decisionmakers and considers the maximum MoSCoW priority for each feature requirement. For example, if three decision-makers assign two Should Have and one Must Have priority to a domain feature requirement, the collective priority becomes Must Have. Moreover, The DSST detects inconsistent values, such as two Must Have and one Could have, and asks decision-makers to revise the considered priories. Figure 5 illustrates an example of the group decision-making process and how the tool supports it.
This tool paper introduces a decision support system (DSST) to accelerate the process of technology selection. The DSST empowers Software Producing Organizations with its discussion platform to perform group decision making and reaching a collective decision. The DSST utilizes the MoSCoW prioritization technique (MoSCoW ) to assess criteria weights and reduce uncertainty, and ISO/IEC quality aspects to indicate the relationship among criteria according to domain experts' knowledge. The experts asserted that the approach increases insight into the selection process and that it reduces the time and cost of the decision-making process. We plan to create a community around the DSST that regularly will update the curated knowledge base with new technology alternatives and features. Additionally, we are looking at methods to automatically extract domain features from manuals and documentation, using text mining techniques.