In contrast to basic items such as movies, books, and songs, configurable items consist of individual subcomponents that can be combined following a predefined set of constraints. Due to the increasing size and complexity of configurable items (e.g., cars and software), a simple enumeration of all possible configurations in terms of a product catalog is not possible. Configuration systems try to identify a solution (configuration) that takes into account both, the preferences of the user and a set of constraints that defines in which way individual subcomponents are allowed to be combined. Due to time limitations, cognitive overloads, and missing domain knowledge, configurator users are in many cases not able to completely specify their preferences with regard to all relevant component properties. As a consequence, recommendation technologies need to be integrated into configurators that are able to predict the relevance of individual components for the current user. In this paper, we show how the determination of configurations can be supported by neural network based recommendation. This approach helps to predict user-relevant item properties using historical interaction data. In this context, we introduce a semantic regularization approach that helps to take into account configuration constraints within the scope of neural network learning. Furthermore, we demonstrate the applicability of our approach on the basis of an evaluation in an industrial configuration scenario (high-voltage switchgear configuration).
1. Introduction In contrast to basic items such as books, movies, and songs, configurable items are composed of subcomponents which must be combined conform to a set of predefined constraints [ 1]. For reasons of combinatorial explosion, it is in many cases impossible to enumerate the individual items (configurations) in terms of a product catalog. Related example domains are automotive [2], software
(e.g., configuration of operating systems) [ 3], and telecommunication infrastructures [4]. Due to the increasing size and complexity of configurable items, it becomes important to integrate recommendation algorithms into configuration processes to support users in component and/or parameter selection.
Informally, configuration can be regarded as a product design activity where the resulting item (also denoted as product or configuration) is composed of elements of a pre-defined set of basic components/parameters [ 1]. In this context, the chosen components must be consistent with a given set of constraints that define restrictions regarding the possible component combinations. On the nEvelop-O LGOBE 3rd Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) & 5th Edition of Recommendation in Complex 0000-0002-1670-7508 (M. Uta); 0000-0003-0108-3146 (A. Felfernig); 0000-0003-0725-7450 (D. Helic) types of decision-theoretic optimizations. An overview of existing integration approaches of recommendation 2. Working Example technologies into configuration systems is provided a.o. in Falkner et al. [13]. Existing integrations focus on a 2-phase process where recommendations of feature set- As a basis for the following discussions on integrating tings are predetermined and then recommended to the neural network based predictions of user preferences, we user. In the case of inconsistent recommendations, alter- first introduce the definition of a configuration task (see native recommendations are calculated repeatedly until Definition 1). a consistent recommendation can be presented. Definition 1 . A configuration task can be defined by a
Compared to existing approaches to the integration tuple ( , , , ) where = { 1, 2, .., } is a set of fiof recommendation algorithms with configuration, we nite domain variables, = {( 1), ( 2), .., ( )} show how to take into account configuration constraints is a set of corresponding domain definitions, and = already in the learning phase and thus minimize the prob- { 1, 2, .., } is a set of rules that define how a configuraability of inconsistent recommendations to be detected in tion can be derived from a given set of customer requirethe subsequent configuration phase. In this paper, we fol- ments = { = , .., = } where elements of low the idea of case-based recommendation [16] where are regarded as variable value assignments. historical configurations with similar parameter settings A simple example of a configuration task definition as those already specified by the current user are used as is the following (see Example 1) where represents a basis for identifying nearest-neighbor configurations. a park distance control feature and represents fuel In our work, we use such a case-based approach as a consumption in gallons/100miles. baseline version. This version is then compared with two diferent versions of a feed-forward neural network based Example 1: Configuration Task . configurator integration. The first version focuses on the • = { , , , , 4 - ℎ , } prediction of configuration parameter settings relevant • = {( ) = for the user. The second version follows the same goal { , , , }, () = { , }, but also takes into account the fact that recommenda- ( ) = {1.7, 2.6, 4.2}, () = tions should be consistent with the underlying constraint { , }, (4 - ℎ ) = { , }, ( ) = set. To support this goal, we propose a semantic regular- { , }} ization of a feed-forward (multi-class and multi-branch) neural network that is used as a configuration parameter • = { 1 ∶ 4- ℎ = → = , 2 ∶ prediction model. = → ≠ , 3 ∶ = 1.7 →
The major contributions of this paper are the follow- = , 4 ∶ = 2.6 ∧ = →
ing. (
Definition 3 . Given the definition of a configuration poses of this example and also for discussing the neural task ( , , , ) , a corresponding recommendation network based recommendation approach, we apply a = { = , .., = } is a set of variable value one hot encoding of the configuration variables, for examassignments of ∈ . A recommendation is consis- ple, in session 1, the configured car is of type city. In the tent if ∪ ∪ is consistent, i.e., a solution can be scenario shown in Table 1, a simple case-based reasonfound. ing recommender would search for one or more nearest neighbors (NN) and recommend the variable settings that
Example 3. = { = , 4 - ℎ = , = were choosen most often by the nearest neighbors. In } our example, the nearest neighbor (session) of the current
Following the approach of case-based reasoning [16], session is session 3 (in terms of the number of equivalent it can be the case that recommended variable value as- variable values). If we assume |NN|=1, we would recomsignments are inconsistent with the already defined user mend = { = , 4 - ℎ = , = } to requirements and the rules (constraints) defined in the the user in the current session (if we intend to predict all knowledge base. This is the case if recommendations unspecified variable values at the same time). are determined from already completed configuration Importantly, since the current user is interested in a sessions without taking into account configuration con- car of type combi which requires the inclusion of a skibag straints (rules in ). In the following, we provide a simple (see Example 1), such a recommendation ( ) induces example of a case-based reasoning approach and then an inconsistency between the user requirements and the focus on how to take into account configuration rules configuration knowledge base (the set of rules ). A in terms of a semantic regularization when optimizing a traditional approach to deal with such a situation is to neural network responsible for recommending variable test the next recommendation for consistency and do this settings. until a consistent recommendation could be found [13]. Our approach (that will be introduced in the following) 3. Recommending Configurable to deal with such a situation is to introduce a semantic regularization into the neural network learning phase Items which helps to avoid inconsistent recommendations as far as possible.
As already sketched in the previous section, recommendations in the context of configuration scenarios are represented by a set of attribute assignments, i.e., a recommendation could include a single attribute setting but also numerous settings recommended at the same time. In this section, we discuss diferent approaches to recommend variable value settings in the context of knowledge-based configuration scenarios.
Neural Network based Recommendation Our basic approach to recommend variable values in the context of rule-based configuration is based on the feed-forward neural network structure depicted in Figure 1. In such networks, the input layer consists of possible values (represented in terms of a one hot encoding) that have already been specified by a user. For example, the variable values (preferences) that have already been specified by the user Case-based Recommendation Table 1 represents a in session are { = , = , = 2.6} . simple example of a set of already completed configu- Networks as those depicted in Figure 1 can be trained rations that can be used as a basis for determining rec- in a domain-dependent fashion on the basis of a dataset ommendations. In this example, configuration sessions type = city =
...
fuel = 1.7
comprised of already completed configuration sessions Constraint-Aware Recommendation For reasons
(see Sessions 1–3 in Table 1). Furthermore, the hidden of potentially inconsistent recommendations, we have
layer is used for learning dependencies between input introduced an enhanced neural network learning phase
values selected by the user and corresponding variable including a semantic regularization where inconsistent
values of potential relevance for the user. The number of recommendations are taken into account as
regularizanodes in the hidden layer is regarded as hyper-parameter tion term. In other words, although parts of the
domainto be optimized in an item domain dependent fashion (in specific rules/constraints can be learned from the
under[17] an equal amount of neurons in the input layer and lying training dataset, it can be the case that some or
the hidden layer showed the best performance). Finally, even many constraints are neglected and the resulting
the output layer supports a multi-branch approach (one variable value recommendations induce an inconsistency.
branch per variable) where each branch is splitted into We denote this approach as constraint-aware neural
net output nodes representing the diferent domain val- works which are extremely relevant in recommendation
ues of variable . In contrast to the input and hidden scenarios where domain-specific constraints/rules have
layer which use a ReLU activation function, classification to be taken into account by the recommender. To
reis implemented using softmax . The choice of the train- duce the probability of inconsistent variable value
recing hyper-parameters has been made based on several ommendations, knowledge base rules/constraints are
test executions. Optimizer “Adam” [18] has shown the taken into account in the learning process. This goal
best performance compared to other gradient decent op- is achieved by integrating the results of a consistency
timizers like “ADAGRAD”, “RMSProp” and “SGD” [19]. check of the proposed recommendation (more
pre“Adam” uses adaptive estimation of first order and sec- cisely, ∪ ∪ ) into a corresponding loss function
ond order moments, which slows down the adjustment as shown in Formula 1.
soeflnecetuerdonpawraemigehttesrtshfeomrtohree “sAtedpasmh”avoeptbimeeinzedrownee.reTahne () ← () + Ω() + × Π() (
In the basic version of our approach, neural networks function, the optimization of our loss function has to
are trained on the basis of training dataset (see, e.g., ses- resort to approximation of gradients via computation of
sions (1–3 in Table 1).The usage scenario of this basic ifnite diferences.
neural network approach is the following: if a user
interacts with a configurator and has already specified a set of User Interaction and Knowledge Representation
initial requirements ( ), the neural network can be ex- Our approach to neural network based variable value
recploited for the recommendation of variable values. Since ommendation for knowledge-based configuration helps
the basic version of the neural network can only learn to reduce the probability of inconsistency-inducing
recconstraints/rules from the available set of completed con- ommendations (see Section 4) and thus also can help to
ifgurations, it can be the case that predictions induce an make configuration processes less time-consuming for
inconsistency with the underlying rule set. users. The proposed recommendation approach is
flexible in the sense that recommendations for single-variable
assignments as well as combined variable assignment
recommendations can be supported. In our working ex- veloped a case-based reasoning approach (see Section 3)
ample, we did not take into account settings, where a that recommends variable value settings on the basis of
configuration task is organized in phases where in each the preferences of the nearest neighbors (in the given
phase a specific subcomponent of the product is config- setting, = 1 achieved the highest prediction quality).
ured (e.g., software configuration as part of the configu- Furthermore, the two versions of the neural network
ration of a whole computer). On the user interface level, based approach have been implemented on the basis of
recommendations are mostly related to variables within the Keras API [21]. 2 The learning of the neural network
a specific phase. However, recommendations can also model is based on 32 iterations during the learning phase
be determined on the basis of existing variable settings of the model where 80% of the data is used for training
from diferent phases. purposes and 20% for testing. The first version of the
neural network model has been trained without taking
Recommendation Consistency The achievable de- into account the rules in the configuration knowledge
gree of recommendation consistency also depends on base whereas the learning of the model for the second
the used knowledge representation. In the case of a rule- version is based on the loss function included in Formula
based knowledge representation [6], it is not always fea- 1. For validation of the models they have been applied
sible to correctly predict if it is possible to complete a separately to 20 configurations which where not part of
partial configuration, i.e., given a (consistent) set of cus- the training or testing data.
tomer requirements, is it possible to find assignments
for the remaining variables in such a way that a consis- Prediction Quality Our first goal was to analyze the
tent and complete configuration can be achieved. If a prediction quality of the three variable value
recommenmore compact representation of all satisfiable variable dation approaches discussed in this paper: (
The configurator application for the high-voltage switchgear domain has been developed with the goal to reduce engineering efort during the ofering stage of these highly complex systems. The underlying dataset includes = 720 complete configurations developed by skilled sales employees from Siemens Energy AG. Each entry of the dataset consists of = 60 attribute settings (assignments), i.e., each configuration is described by 60 variables (representing features and subcomponents). In this context, 10 out of the 60 variables have been defined as basic switchgear features which are assumed to be selected by the user before a recommendation can be triggered (e.g., basic switchgear category to be installed).
The focus of recommendation are the remaining 50 more specific features with a sometimes lower degree of understandability where it is often an issue for users to find good or even optimal settings (e.g., AC supply voltage).
The dataset is composed of consistent configurations that have been built on the basis of a rule-based configuration system.1 As a baseline in our evaluation, we have de|
∩ | | |
(
60
Consistency We were also interested in the degree of
consistency of the determined recommendations ,
i.e., consistent( ∪ ∪ ). The consistency of the
recommendation of a configuration has been
measured in terms of the share of knowledge base consistent 0 n1u0mber of20already k3n0own var4ia0ble valu5e0s 60
predictions ∗ part of the recommendation in relation
to the total number of predictions contained in , see Figure 3: Consistency of high voltage switchgear related
equation 3. As can be seen in Figure 3, the semantic reg- rperaesdoicntiinogn)s. (SR = semantic regularization, CBR = case-based
ularization helps to decrease the inconsistency degree of
recommendations (compared to the CBR and the basic
neural network based approach). Starting with a consis- rators and compared it with cased-based and basic neural
tency of 95.27% (ten variables values already known) the network based recommendation. To improve the
predicneural network based approach with semantic regulariza- tion quality and consistency of recommendations, we
tion (
References [16] B. Smyth, Case-Based Recommendation, in: The Adaptive Web, volume 4321 of LNCS, Springer,
Berlin, Heidelberg, 2007, pp. 342–376. [17] M. Uta, A. Felfernig, Towards machine learning based configuration, in: C. Forza, L. Hvam, A. Felfernig (Eds.), 22nd International Configuration Workshop, 2020, pp. 25–28. [18] D. P. Kingma, J. Ba, Adam: A method for stochastic
optimization, 2017. a r X i v : 1 4 1 2 . 6 9 8 0 . [19] S. Ruder, An overview of gradient descent optimization algorithms, CoRR abs/1609.04747 (2016). URL: http://arxiv.org/abs/1609.04747. [20] H. Andersen, H. Hulgaard, Boolean Expression Diagrams, in: 12th IEEE Symp. on Logic in Computer
Science, IEEE, Warsaw, Poland, 1997, pp. 88–98. [21] F. Chollet, Keras, https://github.com/fchollet/keras, 2015.