Towards Open Configuration Alexander Felfernig1 and Martin Stettinger1 and Gerald Ninaus1 and Michael Jeran1 and Stefan Reiterer2 and Andreas Falkner3 and Gerhard Leitner 4 and Juha Tiihonen5 Abstract. Configuration technologies are typically applied in tion of knowledge engineering tasks by the mentioned groups. We closed settings where one (or a small group of) knowledge engi- denote this approach as community-based knowledge engineering. neer(s) is in charge of knowledge base development and mainte- Suboptimal Decisions. A basic assumption of existing configura- nance. In such settings it is also assumed that only single users con- tion systems is that products and services are typically configured by figure the corresponding products and services. Nowadays, a couple single users. However, many scenarios exist where not a single user of scenarios exist that require more openness: it should be possible but a group of users is in charge of configuring a product (see Sec- to cooperatively develop knowledge bases and to jointly configure tion 3). Existing configuration environments do not take into account products and services, even by adding new features or constraints in such scenarios which often leads to situations where a single user a flexible fashion. We denote this integration of groups of users into has to ”encode” the requirements and preferences of a whole group. configuration-related tasks as open configuration. In this paper we This can lead to suboptimal configurations (decisions) that do not introduce features of open configuration environments and potential reflect the group preferences in an optimal fashion. Future configu- approaches to implement these features. ration technologies should take into account the fact that groups of users can be engaged in configuration processes and provide group decision mechanisms that help the group to jointly configure a prod- 1 Introduction uct in a consensual fashion. We denote this type of configuration Configuration [8, 24, 37] is one of the most successful technologies as group-based configuration. Especially in scenarios where multi- of Artificial Intelligence (AI). It is applied in many domains such ple stakeholders define and configure products, enhanced flexibility as telecommunication [17], furniture [19], and financial services [9]. is required: configurator users may request to add or refine product Most configuration-related functionalities are assuming closed set- features and constraints which can be seen, for example, in open in- tings where knowledge bases are developed by a single (or a small novation [4] or postponement scenarios [18, 42]. We subsume such group of) knowledge engineer(s) and the corresponding configura- activities under the term flexible product enhancement. tors are applied by single users. Implementing configurator applica- The concepts of community-based knowledge engineering, group- tions this way entails drawbacks which become manifest in terms of based configuration, and flexible product enhancement can be scalability problems in knowledge engineering [33] and suboptimal summed up under the notion of open configuration. In this paper we decisions if a single user decides for the whole group [16]. sketch functionalities which have to be provided by open configura- Scalability Problems. The transformation of domain knowledge tion environments. In Section 2 we introduce features and potential into a configuration knowledge base is an effortful process of- technological solutions to tackle the issue of scalability in knowledge ten characterized by a knowledge acquisition bottleneck [20] that engineering scenarios. In Section 3 we discuss features of group- is considered as a major obstacle for a sustainable application of based configuration. In Section 4 we discuss aspects of product en- knowledge-based technologies [21, 41]. To tackle this bottleneck, hancement in open configuration. With Section 5 we provide a dis- efficient approaches have been developed that support graphical cussion of related work. We conclude the paper with Section 6. knowledge engineering [7, 22] and intelligent debugging [6, 14, 35]. These approaches help to improve the efficiency of knowledge en- 2 Community-based Knowledge Engineering gineering but still do not solve the problem of missing scalability: the increasing amount and complexity of configuration knowledge In the following we will discuss aspects that become relevant if we bases exceeds the resources available for performing the correspond- want to integrate a larger group of users into configuration knowledge ing development and maintenance operations [23, 33]. In order to engineering. For the sake of simplicity and without loss of general- assure scalability, future configuration technologies have to support ity we assume that a configuration knowledge base is represented in a deeper integration of a wider group of users (e.g., product devel- terms of a constraint satisfaction problem (CSP) [27] consisting of opers, marketing experts, sales representatives, and knowledge en- a set of variables V = {v1 , ..., vn } with corresponding domain def- gineers) into knowledge engineering. Related solutions should go initions (dom(vi )), and a set of constraints C = {c1 , ..., cm }. We beyond state-of-the-art approaches that are focusing on experienced base our discussions on the following simplified financial services knowledge engineers and programmers [24] by allowing the comple- configuration knowledge base. 1 TU Graz, Austria, email: {firstname.lastname}@ist.tugraz.at 2 SelectionArts, Austria, email: stefan.reiterer@selectionarts.com • V = {willingness to take risks (wr), expected return rate (rr), in- 3 Siemens, Austria, email: andreas.a.falkner@siemens.com vestment period (ip)} 4 University of Klagenfurt, Austria, email: gerhard.leitner@aau.at • dom(wr)= {low, medium, high}, dom(rr)={<6%, 6-9%,>9%}, 5 Aalto University, Finland, email: juha.tiihonen@aalto.fi dom(ip) = {shortterm, mediumterm, longterm} micro task topic description variables definition/evaluation of variables included in V questions definition/evaluation of questions related to vi ∈ V dialog sequences definition/evaluation of question sequences constraints definition/evaluation of constraints in C examples definition/evaluation of test cases in T diagnoses evaluation of conflict resolution alternatives for C Table 1. Community-based knowledge engineering: example micro tasks. • C = {c1 : wr = medium → ip 6= shortterm, with questions posed to the user of a configurator application – al- c2 : wr = high → ip = longterm, ternative formulations of such questions and also the sequences in c3 : ip = longterm → rr = <6% ∨ rr = 6-9%, which these questions are posed should be defined and evaluated by c4 : rr = >9% → wr = high, users. In addition to structural properties typically defined in terms c5 : rr = 6-9% → wr 6= low ∧ wr 6= medium} of variables or component types and their relationships, constraints define additional restrictions on possible combinations of variable In cases where one or a small group of knowledge engineers is values (components). in charge of developing and maintaining a configuration knowledge Especially in community-based scenarios, where a larger number base, attributes (component types), domains, and related constraints of users interacts with the knowledge engineering environment, en- are typically formalized on the basis of examples and textual de- gineering practices will change in the sense that users are providing scriptions provided by domain experts [24]. If the product domain knowledge chunks in a collaborative fashion and the knowledge en- knowledge has to be adapted, the whole process is restarted, i.e., do- gineering environment is in charge of aggregating this information. main experts articulate the change requests in an informal fashion In this context, it is necessary to have mechanisms that automatically and knowledge engineers implement the needed adaptations. distribute knowledge acquisition tasks among users in a systematic The correctness of changes performed on a knowledge base can be fashion (e.g., depending on the workload, knowledge level, and pref- evaluated, for example, on the basis of regression tests where positive erences of users). Such tasks can be represented in a more-or-less and negative test cases are used to figure out whether the knowledge traditional form of todo-lists but can also be represented in terms of base shows the intended behavior [6]. Positive test cases (examples) so-called games with a purpose [40] which is an upcoming trend also are a specification of an intended behavior of the knowledge base in the knowledge engineering field [36]. and negative test cases exemplify unintended behavior. Existing ap- A simple example of such a knowledge acquisition interface is proaches to configuration knowledge base testing and debugging ex- depicted in Figure 1. In this example game, the users Ann and Paul ploit positive test cases to detect errors/deficiencies by inducing con- have the task to cooperatively figure out combinations of customer flicts in the incorrect configuration knowledge base. Such conflicts requirements that are incompatible, i.e., induce an inconsistency with are minimal sets of constraints that are responsible for the faulty be- the knowledge base. The players have successfully completed their havior of the knowledge base and therefore have to be adapted by task if they, for example, selected the same set of assignments as knowledge engineers. candidates for incompatibilities. The underlying assumption of this Community-based Knowledge Engineering. Intelligent testing and game is that Ann does not know the input of Paul and vice-versa. debugging [6] is an important contribution to the improvement of Further examples of gamification-based interfaces for configura- knowledge engineering processes. However, the growing size and tion knowledge acquisition are: cooperative definition of relevant complexity of configuration knowledge bases often makes it hard for variables (including their domains), the estimation of intuitive di- individual knowledge engineers to keep track of new developments alog sequences (which questions should be asked in which order), and adaptations. As a consequence, more time is needed to provide the derivation of further constraint types (e.g., filter constraints that a new production version of the configuration knowledge base and match user requirements to corresponding technical product prop- the probability of including erroneous constraints increases. In or- erties), and the estimation of accepted repair rankings in situations der to assure scalability, it is important to integrate end-users more where no solution could be found. Such scenarios can be supported deeply into knowledge base development and maintenance and thus by input templates that represent micro-tasks (see Figure 1). to exploit unemployed knowledge engineering potentials. Testing and Debugging. The definition and evaluation of (posi- In the following we discuss issues that have to be taken into ac- tive and negative) test cases is a crucial issue since the correctness count when integrating groups into community-based knowledge en- of a test suite directly influences the correctness of the results de- gineering processes. An in-depth integration of a larger group of termined by a configurator. In [6] positive and negative examples are users allows knowledge engineers to delegate basic engineering tasks exploited for debugging knowledge bases on the basis of the concepts (so-called micro tasks). Table 1 provides an overview of micro task of model-based diagnosis [32]. In this context, positive examples are topics. For each topic a couple of different concrete micro tasks can exploited for inducing conflicts in a configuration knowledge base. be defined, for example, a variable can be defined but also evaluated A negative example is assumed to be integrated in negated form into with regard to the appropriateness of it’s domain definition. the knowledge base in the case that it has not been rejected by the In order to figure out variables (component types) relevant for the knowledge base. On the basis of the following two test cases (exam- configuration knowledge base, users should be allowed to enter pro- ples) we can show how positive examples are used to find errors in posals for variables and component types (including the correspond- the knowledge base. Both test cases are in conflict with constraints ing domain definitions) on their own. Variables are often associated Figure 1. Sketch of a user interface for game-based knowledge acquisition. The overall goal of the game is that both players agree on the set of incompatible value combinations of a given set of variables. This user interface can be regarded as a micro task template for the acquisition of incompatibility constraints. Figure 2. Group-based diagnosis of a faulty configuration knowledge base. Diagnoses are selected by taking into account the expertise of users/knowledge engineers: the higher the personal score (value derived from his/her personal contributions), the higher the weight given to his/her opinion. in the configuration knowledge base introduced in Section 2. solving the conflict, i.e., c5 must be selected. Overall, the elements in the diagnosis D2 = {c3 , c5 } have a lower community support and • t1 : wr = high ∧ rr = >9% therefore will be changed or deleted by the users in order to restore • t2 : rr = 6-9% ∧ wr = medium the consistency with the test-suite {t1 , t2 }. A conflict between a test case t and a set of constraints in the con- figuration knowledge base can be defined as a conflict set CS ⊆ C: 3 Group-based Configuration CS ∪ t inconsistent. Such a conflict set CS is minimal if there does not exist another conflict set CS 0 with CS 0 ⊂ CS. To resolve a An assumption of existing configuration environments is that there minimal conflict, only one element has to be deleted from CS. In is no need for additional configuration support in scenarios where our example, the test case t1 is in conflict with the constraints c2 and groups of users are jointly configuring their preferred product or ser- c3 and test case t2 is in conflict with the constraint c5 . Consequently vice. A major consequence of this assumption is that single users are we have two different (and minimal) conflict sets which are CS1 : forced to encode the preferences of a group which is often done in a {c2 , c3 } and CS2 : {c5 }. Resolving these conflicts results in two dif- suboptimal fashion. ferent diagnoses, namely D1 = {c2 , c5 } and D2 = {c3 , c5 }, i.e., a Within the scope of an industry study with representatives of N=25 diagnosis is a hitting set [32] which includes at least one constraint companies applying configurators we figured out that none of the ex- from each of the given conflict sets. isting configuration environments provides technologies that support Typically, there are many alternative diagnoses and the question groups of users in jointly configuring a solution. However, there is has to be answered which of these is acceptable for the users en- a strong agreement on the fact that such technologies have to be in- gaged in testing and debugging. Figure 2 depicts a basic approach cluded in future configurators. The study participants reported dif- of integrating knowledge about the users expertise in the determina- ferent scenarios for the application of group-based (socially aware) tion of a diagnosis. For the conflict CS1 = {c2 , c3 }, the majority of configuration technologies. Social awareness in this context denotes users prefers to keep c2 as-is and to delete or change c3 to resolve the fact that specific properties of group decision processes are ex- the conflict. Since CS2 is a singleton, no alternatives exist for re- plicitly taken into account by the configuration environment (e.g., ID domain for group-based configuration components and constraints decision makers requirements, releases, 1 software release plans stakeholders in software project dependencies, preferences (new) features, constraints between representatives from different 2 product line scoping and open innovation features, preferences departments, customers (new) destinations, hotels, bundle configuration (e.g., hotel, flight, 3 sightseeing tours, (resource) travel group tour, etc.) constraints, preferences (new) persons, constraints stakeholder selection for a new software 4 regarding competences and (initial) team members project resources, preferences components, interfaces, architectural design in software (distributed) software project 5 technologies, constraints between development members components, preferences financial services, resource 6 financial service configuration family members constraints, preferences rooms, furniture, light control building configuration (e.g. smart home, family members, suppliers, 7 equipment, constraints between office block) company representatives components, preferences project proposals, resource evaluators, consultants, decision 8 funding decisions constraints, preferences makers Table 2. Application scenarios for group-based configuration identified within the scope of a study with N=25 companies applying configuration systems. the need to achieve consensus among group members). Examples of A major issue for future research is the consideration of longer such scenarios are depicted in Table 2. time periods. For example, if a group of friends jointly configures In these scenarios a group of users is in charge of jointly config- a holiday trip every year, the aggregation mechanisms used by the uring a product or service, for example, when configuring a holiday group-based configuration environment should take into account (as trip (bundle configuration) for a group of friends [25], the require- far as possible) the degree to which individuals had to accept trade- ments and preferences of all group members should be taken into offs in the past and use this information for the recommendation of account. When configuring a software release plan, the preferences fair trade-offs in future configuration sessions. of individual stakeholders regarding the assignment of requirements On the technical level the above mentioned properties require ba- to releases have to be taken into account [31]. sic research in the following areas. Taking into account requirements and preferences of group mem- First, constraint-based search methods have to be extended with bers requires decisions regarding trade-offs. In the context of holiday mechanisms that help to predict (partial) configurations which are of trips such a trade-off could be the acceptance of a lower-quality ho- relevance for the group. This requires learning methods for search tel which is much nearer to the sightseeing destination preferred by heuristics [34] that help to predict relevant configurations in an ef- a specific user. When configuring software release plans, a trade-off ficient fashion. Furthermore, it is important that configurators are could concern the postponement of a specific requirement to a later able to determine similar and diverse configurations efficiently which release while increasing the importance level of this requirement (to could also be achieved on the basis of the mentioned heuristics. avoid further postponements). Second, the determination of trade-offs for inconsistent require- The determination of trade-offs must be based on preference ag- ments and preferences has to be based on efficient diagnosis meth- gregation mechanisms [29] that take into account the preferences of ods integrated with intelligent preference aggregation mechanisms all group members as far as possible. For example, the least misery [29] that can help to better predict trade-offs acceptable for all group strategy avoids massive discriminations of individual group members members. These aggregations must take into account the histories by minimizing the maximum number of trade-offs to be accepted by stored in interaction logs in order to guarantee decision fairness in an individual. In contrast, majority voting follows the opinions of the the long run. majority of the group members which can lead to discriminations Third, negotiation and argumentation mechanisms have to be de- against individuals. veloped which support individuals to express acceptable trade-offs. An example of the application of the least misery strategy in the In our holiday configuration scenario an example of such a statement context of deciding about a common sightseeing trip is depicted in is ”I accept to visit Greece this year if we agree to organize a trip to Table 3. In this simplified example, each person is allowed to select Italy next year”. Such arguments cannot be expressed on the basis of at most two destinations and the corresponding trip must include two existing preference representations. destinations. Since Ben and John have similar preferences, majority voting would discriminate Kate. In contrast, least misery tries to find 4 Flexible Product Enhancement a trade-off that has the potential to create group consensus. For a detailed discussion of preference aggregation mechanisms we refer The ability to include additional variables (component types), values the reader to [29]. (components), and constraints in a flexible fashion is important for the implementation of open configuration. destination Lindwurm Großglockner Pyramidenkogel Isonzo Valley Ben 1 1 0 0 John 1 1 0 0 Kate 0 0 1 1 least misery 1 0 1 0 majority voting 1 1 0 0 Table 3. Example set of tourist destinations (in the Alps-Adriatic area). The assumption in this example is that each person is allowed to articulate at most two preferences and the trip must include at least two destinations. Product line scoping [26] (in the context of software product line of constraint sets. In this context, knowledge bases are learned on the engineering) is in the need of such a flexibility since the features and basis of positive and negative examples. Generated examples are pre- constraints element of the product line are not completely predefined sented to users who have to decide whether the examples are positive at the beginning of the engineering process. A larger group of users or negative. Learning is based on a so-called bias that is a knowledge has to jointly decide which components (features) and constraints base generated from a vocabulary (variables, domains, and opera- should be part of the product line. Thus, product line scoping can tors). The bias is systematically reduced on the basis of the infor- be interpreted as open configuration where new alternatives and con- mation included in the examples, for instance, all conflicts induced straints (and preferences) can be integrated within the scope of the in the bias by a positive example have to be resolved. In the case configuration (product line scoping) process. of a negative example, at least one conflict must be preserved which Open innovation [4] reflects the idea of integrating customer com- guarantees the rejection of the negative example. Approaches to the munities into new product development processes of a company. In application of association rule mining for configuration knowledge this context, variability modeling for product lines also requires the discovery are discussed in [23]. An important research issue in this support of an easy integration of new component types, components, context is to assure the understandability and manageability of the and constraints which reflect features to be supported by future prod- derived configuration knowledge [12]. ucts. In both scenarios, the integration of new items has to be sup- Human Computation is based on the idea of passing those tasks to ported by corresponding group decision processes (see Section 3), humans which are easy to solve for them but are not solvable by com- for example, before a new feature is integrated into the model, the puters [39]. Related research has already been conducted in the areas group has to perform the needed validation steps and decide about of ontology construction (concept learning) [36] and sentiment anal- the inclusion of the feature. This also holds for the afore mentioned ysis in text documents [30]. A major idea of the work presented in scenarios of release planning and holiday trip configuration. this paper is to exploit the concepts of Human Computation as a cen- A further example of the need for flexible enhancements are post- tral mechanism for configuration knowledge base construction and ponement strategies [18, 42]. An example is the automotive indus- maintenance. These mechanisms go beyond concept learning [36] try, where basic car configurations are delivered to dealers who can and include tasks such as diagnosis discrimination, test case classifi- then integrate additional components such as MP3 players and tow- cation and evaluation, and configuration dialog design. bars, i.e., are enabled to integrate their own products and services Preferences are not known beforehand but are constructed within into the basic configuration delivered by car producers. Conform to the scope of a decision process [3, 38]. As a result, biases occur the definition given in [18], the mentioned scenario is of type-III which often lead to suboptimal decisions. Concepts to deal with where customers are allowed to specify additional equipment when (group) decision problems in recommender systems are discussed in they already have a more precise idea of the interior of the car. The [11, 15, 25, 28, 31]. A major issue for future research in this context corresponding configuration model has to provide flexible interfaces is an in-depth investigation of decision biases in group decision mak- that allow an easy integration of new component types, components, ing. An important question is to which extent biases are compensated and constraints. A knowledge representation concept that can be ex- or become more intense when groups decide. ploited in this context are contextual models [10] which allow a sys- tematic extension of existing base diagrams with additional items relevant in a specific context (e.g., the car dealer context). In such 6 Conclusions scenarios, developers of configurator solutions also have to take into account that – depending on the additional items introduced – search In this paper we introduced central ideas and research questions re- heuristics [34] have to be adapted in order to assure efficient search. lated to open configuration. Openness in this context is related to the idea of a closer integration of end-users into configuration knowledge base development and maintenance operations and of supporting de- 5 Related and Future Work cision processes in scenarios where groups of users are in charge of configuring a product or service. Furthermore, open configuration is Intelligent testing and debugging methods for configuration knowl- often characterized by the need of being able to integrate new items edge bases have been introduced in [6] where positive test cases (e.g., component types, components, and constraints) ”on the fly”. can detect errors by inducing conflicts in a configuration knowledge On the basis of the results of a first industry study we reported exam- base. Conflicts are then resolved on the basis of model-based diag- ple application domains and discussed related research challenges. nosis [32]. In open configuration scenarios, testing and debugging The concepts presented in this paper can be applied in a broad range approaches have to be adapted to group-based settings where diag- of scenarios which go beyond open configuration. Further example nosis discrimination has to take into account group preferences. application domains are (constraint-based) scheduling [1], recom- Bessiere et al. [2] introduced basic mechanisms to the learning mender systems [5], and utility evaluation where user groups are in charge of evaluating alternatives [13]. [19] A. Haag, ‘Sales Configuration in Business Processes’, IEEE Intelligent Systems, 13(4), 78–85, (1998). [20] F. Hayes-Roth, D. Waterman, and D. Lenat, Building Expert Systems, ACKNOWLEDGEMENTS Addison-Wesley, 1983. [21] S. Hoppenbrouwers, P. Lucas, D. Romano, and D. Moffat, ‘Attacking The work presented in this paper has been conducted in the research the knowledge acquisition bottleneck trough Games-For-Modelling’, in AISB Symposium, pp. 81–86, (2009). project P EOPLE V IEWS funded by the Austrian Research Promotion [22] L. Hotz, A. Felfernig, M. Stumptner, A. Ryabokon, C. Bagley, and Agency (843492). K. Wolter, ‘Configuration Knowledge Representation & Reasoning’, in Knowledge-based Configuration – From Research to Business Cases, eds., A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen, chapter 6, 59– REFERENCES 96, Morgan Kaufmann Publishers, (2013). [23] Y. Huang, H. Liu, W. Ng, W. Lu, B. Song, and X. Li, ‘Automat- [1] P. Baptiste, C. Le Pape, and W. Nuijten, Constraint-based Scheduling, ing knowledge acquisition for constraint-based product configuration’, Kluwer, 2001. Journal of Manufacturing Technology Management, 19(6), 744–754, [2] C. Bessiere, R. Coletta, B. O’Sullivan, and M. Paulin, ‘Query-driven (2008). constraint acquisition’, in 21st International Joint Conference on Arti- [24] L. Hvam, N. Mortensen, and J. Riis, Product Customization, Springer, ficial Intelligence (IJCAI’07), pp. 50–55, Hyderabad, India, (2007). 2007. [3] J. Bettman, M. Luce, and J. Payne, ‘Constructive consumer choice pro- [25] A. Jameson, S. Baldes, and T. Kleinbauer, ‘Two methods for enhancing cesses’, Journal of Consumer Research, 25(3), 187–217, (1998). mutual awareness in a group recommender system’, in International [4] H.W. Chesbrough, Open Innovation. The New Imperative for Creating Working Conference on Advanced Visual Interfaces, pp. 447–449, Gal- and Profiting from Technology, Harvard Business School Press, Boston, lipoli, Italy, (2004). 2003. [26] I. John, J. Knodel, T. Lehner, and D. Muthig, ‘A practical guide [5] A. Felfernig D. Jannach, M. Zanker and G. Friedrich, Recommender to product line scoping’, in Software Product Line Conference 2006 Systems – An Introduction, Cambridge University Press, 2010. (SPLC2006), pp. 3–12, (2006). [6] A. Felfernig, G. Friedrich, D. Jannach, and M. Stumptner, [27] A. Mackworth, ‘Consistency in Networks of Relations’, Artificial Intel- ‘Consistency-based diagnosis of configuration knowledge bases’, Ar- ligence, 8(1), 99–118, (1977). tificial Intelligence, 152(2), 213–234, (2004). [28] M. Mandl, A. Felfernig, E. Teppan, and M. Schubert, ‘Consumer Deci- [7] A. Felfernig, G. E. Friedrich, and D. Jannach, ‘UML as Domain Spe- sion Making in Knowledge-based Recommendation’, Journal of Intel- cific Language for the Construction of Knowledge-based Configuration ligent Information Systems (JIIS), 37(1), 1–22, (2010). Systems’, International Journal of Software Engineering and Knowl- [29] J. Masthoff, ‘Group Recommender Systems: Combining Individual edge Engineering, 10(4), 449–469, (2000). Models’, Recommender Systems Handbook, 677–702, (2011). [8] A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen, Knowledge-based [30] C. Musat, A. Ghasemi, A., and B. Faltings, ‘Sentiment Analysis Using Configuration – From Research to Business Cases, Elsevier/Morgan a Novel Human Computation Game’, in 3rd Workshop on the People’s Kaufmann Publishers, 2014. Web Meets NLP, pp. 1–9, (2012). [9] A. Felfernig, K. Isak, K. Szabo, and P. Zachar, ‘The VITA Financial [31] G. Ninaus, A. Felfernig, M. Stettinger, S. Reiterer, G. Leitner, Services Sales Support Environment’, in AAAI/IAAI 2007, pp. 1692– L. Weninger, and W. Schanil, ‘IntelliReq: Intelligent Techniques for 1699, Vancouver, Canada, (2007). Software Requirements Engineering’, in 21st European Conference on [10] A. Felfernig, D. Jannach, and M. Zanker, ‘Contextual Diagrams as Artificial Intelligence / Prestigious Applications of Intelligent Systems structuring mechanisms for designing configuration knowledge bases in (PAIS 2014), p. to appear, Prague, Czech Republic, (2014). UML’, in 3rd International Conference on the Unified Modeling Lan- [32] R. Reiter, ‘A theory of diagnosis from first principles’, Artificial Intel- guage (UML2000), number 1939 in LNCS, pp. 240–254, (2000). ligence, 32(1), 57–95, (1987). [11] A. Felfernig, W. Maalej, M. Mandl, F. Ricci, and M. Schubert, ‘Recom- [33] M. Richardson and P. Domingos, ‘Building Large Knowledge Bases by mendation and decision technologies for requirements engineering’, in Mass Collaboration’, in 2nd Intl. Conference on Knowledge Capture ICSE 2010 Workshop on Recommender Systems in Software Engineer- (K-CAP03), pp. 129–137, (2003). ing, pp. 1–5, Cape Town, South Africa, (2010). [34] T. Schrijvers, G. Tack, P. Wuille, H. Samulowitz, and P. Stuckey, [12] A. Felfernig, S. Reiterer, M. Stettinger, F. Reinfrank, M. Jeran, and ‘Search combinators’, Constraint Journal, 18(2), 269–305, (2013). G. Ninaus, ‘Recommender Systems for Configuration Knowledge En- [35] M. Schubert, A. Felfernig, and M. Mandl, ‘FastXPlain: Conflict De- gineering’, in Workshop on Configuration, pp. 51–54, Vienna, Austria, tection for Constraint-Based Recommendation Problems’, in Trends in (2013). Applied Intelligent Systems (proc. of 23rd International Conference on [13] A. Felfernig, S. Schippel, G. Leitner, F. Reinfrank, K. Isak, M. Mandl, Industrial Engineering and Other Applications of Applied Intelligent P. Blazek, and G. Ninaus, ‘Automated Repair of Scoring Rules in Systems, IEA/AIE 2010), eds., Nicolás Garcı́a-Pedrajas, Francisco Her- Constraint-based Recommender Systems’, AI Communications, 26(2), rera, Colin Fyfe, JoséManuel Benı́tez, and Moonis Ali, volume 6096 15–27, (2013). of Lecture Notes in Computer Science, 621–630, Springer, Cordoba, [14] A. Felfernig, M. Schubert, and C. Zehentner, ‘An efficient diagnosis al- Spain, (2010). gorithm for inconsistent constraint sets’, Artificial Intelligence for En- [36] K. Siorpaes and M. Hepp, ‘Games with a Purpose for the Semantic gineering Design, Analysis and Manufacturing (AI EDAM), 26(1), 53– Web’, IEEE Intelligent Systems, 23(3), 50–60, (2008). 62, (2012). [37] M. Stumptner, ‘An Overview of Knowledge-based Configuration’, AI [15] A. Felfernig, E. Teppan, and B. Gula, ‘Knowledge-based recommender Communications, 10(2), 111–126, (1997). technologies for marketing and sales’, International Journal of Pat- [38] E. Teppan and A. Felfernig, ‘Asymmetric Dominance- and Com- tern Recognition and Artificial Intelligence (IJPRAI), 21(2), 333–354, promise Effects in the Financial Services Domain’, in IEEE In- (2006). Special issue of Personalization Techniques for Recommender ternational Conference on E-Commerce and Enterprise Computing Systems and Intelligent User Interfaces. (CEC/EEE2009), pp. 57–64, Vienna. Austria, (2009). [16] A. Felfernig, C. Zehentner, G. Ninaus, H. Grabner, W. Maaleij, [39] L. von Ahn, ‘Human Computation’, in Technical Report CMU-CS-05- D. Pagano, L. Weninger, and F. Reinfrank, ‘Group Decision Support for 193, Carnegie Mellon University, School of Computer Science, (2005). Requirements Negotiation’, in Advances in User Modeling, Springer [40] L. von Ahn, ‘Games with a Purpose’, IEEE Computer, 39(6), 92–94, Verlag, volume 7138 of Lecture Notes in Computer Science, pp. 105– (2006). 116, (2012). [41] C. Wagner, ‘Breaking the Knowledge Acquisition Bottleneck through [17] Gerhard Fleischanderl, Gerhard E. Friedrich, Alois Haselböck, Herwig Conversational Knowledge Management’, Information Resources Schreiner, and Markus Stumptner, ‘Configuring large systems using Management Journal, 19(1), 70–83, (2006). generative constraint satisfaction’, IEEE Intelligent Systems, 13(4), 59– [42] B. Yang and N. D. Burns, ‘Implications of postponement for the sup- 68, (1998). ply chain’, International Journal of Production Research, 41(9), 2075– [18] C. Forza, F. Salvador, and A. Trentin, ‘Form postponement effects on 2090, (2003). operational performance: a typological theory’, International Journal of Operations and Production Management, 28, 1067–1094, (2008).