=Paper=
{{Paper
|id=Vol-2068/wii1
|storemode=property
|title=On Studying Bad Practices in Configuration UIs
|pdfUrl=https://ceur-ws.org/Vol-2068/wii1.pdf
|volume=Vol-2068
|authors=Tony Leclercq,Maxime Cordy,Bruno Dumas,Patrick Heymans
|dblpUrl=https://dblp.org/rec/conf/iui/LeclercqCDH18
}}
==On Studying Bad Practices in Configuration UIs==
https://ceur-ws.org/Vol-2068/wii1.pdf
On Studying Bad Practices in Configuration UIs
Tony Leclercq, Maxime Cordy, Bruno Dumas, Patrick Heymans
University of Namur, Belgium
first.last@unamur.be
ABSTRACT
In today’s businesses, configurators are essential tools that al-
low customers to personalise a product to their specific needs.
Being often the first contact between the retailer and the cus-
tomer, it is important for a configurator to provide a smooth
user experience. However, these software have been studied
mostly from the reasoning point of view, that is, how to rep-
resent the logical relations between the configuration options
and how to automatically ensure that the user makes no con-
figuration error. In this paper, we focus instead on HCI and
analyse the occurrence of 11 design flaws that occur in 28 car
configurators. These flaws includes violations of general HCI
principles as well as lacks of essential, configurator-specific Figure 1. Some car configurator
functionalities. Our results indicate that most of the studied
configurators contain defects in their UIs including, surpris-
ingly, violations of some basic principles of HCIs. Still, the car configurator where customers can choose between various
most frequent problems are inappropriate implementations of options like trim, motor, colour, accessories, etc. Beneath the
configurator-specific functionalities. user interface lies an intelligent system that provides effec-
tive guidance by preventing errors and providing automated
ACM Classification Keywords support for the configuration task [5, 8]. An error is typically
H.5.2. User Interfaces: Graphical user interfaces (GUI), Con- an attempt to include incompatible options or parameter val-
figurator, Evaluation, Intelligent system ues in the configuration. When the user makes an error, the
configurator should alert her and explain why the configu-
INTRODUCTION ration is invalid. On the other side, when previous choices
The avenue of mass customisation – an established marketing make mandatory to select or to exclude an option, or when
and engineering paradigm that unifies mass production and a parameter accepts only one value, the configurator should
customisation [11] – has yielded new habits in customers’ propagate (i.e., force) this choice so as to simplify the task of
behaviour. In constant search of the ideal product (i.e. good or the user. For instance, selecting a comfort trim automatically
service) that satisfies their particular needs, they hunger for the includes a cruise control system; see Figure 2). It is again
freedom of tailoring sellers’ catalog into a personalised offer. recommended to explain the cause of the propagation, in case
It is therefore not surprising that personalisation is considered the user disagrees with this choice. When errors or undesired
as one of the hottest topics in the retail industry [13]. Still, to propagations occur, the configurator should help repair the
be successful personnalisation must be supported by a smooth configuration [19, 20], that is, automatically compute what
user experience (UX) that offers the highest customisation must be changed to fix the error or undo the propagation, re-
capability while hiding the inherent complexity [12]. spectively. An example of repair system is shown in Figure 3.
Configurator is the typical software support for mass customi- We see that the selected wheels require a particular suspension
sation. It consists in an interactive application where the user system and specific tires, as well as removing the off-road
specifies its requirements by selecting options and setting pa- accessory pack.
rameter values. As a result she obtains the relevant products Configuration knowledge refers to the set of rules that deter-
that can address these needs. For example, Figure 1 shows a mine what is a valid configuration. It originates from rational
(technical, legal, mathematical) and subjective (marketing, aes-
thetics) knowledge. The acquisition of this knowledge is a
critical analysis activity for a successful configurator [2, 6], as
erroneous or incomplete knowledge can have drastic conse-
quences on the products that will be built afterward. Once the
configuration knowledge is defined, one can develop the back-
©2018. Copyright for the individual papers remains with the authors. end part of the configurator responsible for checking errors, as
Copying permitted for private and academic purposes. well as computing propagations and repairs. A standard way
WII’18, March 11, 2018, Tokyo, Japan
Figure 4. Violations of basic ergonomic guidelines: empty controls,
Figure 2. Propagation of mandatory option empty boxes and wrong control placements
commonly provide a high number of options (which makes
both presentation and reasoning challenging), they are directly
aimed at customers (which makes UX a primary concern), and
we can find very different UI design choices across different
brands and countries. More precisely, we check whether 11
UX design flaws occur in 28 configurators we selected across
different brands and countries. These 11 flaws comprise 4
general HMI anti-patterns, 4 specific to navigation-based in-
terfaces, and 3 related to configurator-specific functionalities.
Figure 3. An instance of repair where some options must be added and
Our results reveal that 75% of the configurators show more
others removed than three types of UX flaws, half of them more than six. This
is mostly due to missing or badly implemented configuration-
specific functionalities, although we also observe the violation
to do that is to implement it from scratch in an established of general HCI principles. This corroborates the need for HCI
programming language like Java or PHP. However, this results guidelines specific to configurator engineering.
in scattering the knowledge across the code base, which leads
to increased risks of bugs and poor maintainability [1]. An AN OVERVIEW OF UX ISSUES IN CONFIGURATORS
alternative solution is to encode it as logical rules in a configu- To carry out our study, we select a set of car configurators
ration engine [14, 7, 3], i.e., a dedicated software component amongst the ten top-selling car manufacturers. For each man-
that implements generic algorithms to reason on configura- ufacturer we choose the best selling car model, and for every
tion knowledge. Such engines thus provide the configuration model we consider the configurator of the US, UK and Bel-
functionality at no cost and regardless of the actual application gium websites. This leads us to 28 configurators to evaluate
domain. (as two of the 30 candidates do not allow one to select any
option).
While efficient configuration engines exist, they are only a part
of the challenges in configurator engineering. Indeed, configu- Looking at these configurators, we rapidly note that basic er-
rators belong to a class of interactive applications where UX gonomic guidelines [18, 15, 16, 10] are not satisfied. Indeed,
is of paramount importance [16] and relies on specific func- we observe occurrences of empty controls, empty boxes or
tionalities of these intelligent systems [12]. However, there wrong control placements; see Figure 4. Inappropriate choice
exist neither established nor de-facto HMI standards for con- of controls, inconsistent semantics grouping, and lack of em-
figurators [17, 9]. Even worse, recent studies [1] showed that phasis were also very common. This is relatively surprising,
many public configurators do not follow general HMI guide- as car configurators are popular and powerful marketing tools.
lines, resulting in weak usability. This reveals that designing This makes us consider general HCI principles as our first
HMI guidelines for configurators is a essential element of the evaluation criteria. Our objective being more focused than a
envisioned “body of knowledge dedicated to the engineering general-purpose HCI evaluation, we settle for only four viola-
of configurators” [1, 3]. tions we observed the most: absence of feedback, information
overload, too many controls and bad ergonomics (see more in
A prerequisite to creating these guidelines is to understand
the next section).
what existing configurators do wrong. Therefore, pursuing
previous research [12, 1], in this paper we analyse to what In addition to these general considerations, we also observe
extent existing configurators contain HCI design flaws. We issues related to the management of navigation within the
focus more particularly on car configurators, the most popu- UI. Figure 5 is a striking example that highlights multiple
lar kind of configurators according to the Cyledge’s database problems. First, we see that the interface does not display the
statistics [4]. Car configurators are excellent candidates: they step currently completed. Also it provides no visible way to
Figure 5. All signs of navigation have disappeared
Figure 6. Invalid configuration: multiple wheel sizes are selected
go to the previous or the next step. The user has no idea of
the progress she has already accomplished, and can neither • Bad ergonomics. The configurator does not follow estab-
consult nor preview the current state of her configuration. All lished ergonomics guidelines. These include, e.g., mis-
these navigation-related issues constitute our second group of placed controls and inappropriate colour or font size.
criteria.
Navigation. Configuration being an interactive process, its
Finally, we observe the lack of configuration-specific func- navigation must be absolutely clear. We can see navigation
tionalities that are, however, deemed essential. For instance, from three points of view, i.e. in the UI, in the progress in the
Figure 6 depicts a configuration where three different sizes of process, and in the configuration state. This leads us to four
wheels are selected. Errors are thus unchecked in this configu- types of violations:
rator. Configuration being a long and error-prone process, it
is unrealistic not to help users complete their task. Explana- • Unknown UI location. The user is lost within the UI and
tion of errors, propagations and repairs – as presented in the has no idea what step she is currently completing.
previous section – are other facilitating functionalities. We • Irregular navigation control. In the UI, the user cannot
argue that all configurators should implement them. Together see the control allowing her to move to the next step, be-
with error checking, they are thus part of our third type of cause their position or shape change.
evaluation criteria. All our criteria are properly defined in the • Unknown progress. The progress in the process, i.e. what
next section. percentage of effort the user has already put in or remains
to be done, is not displayed.
• Unknown configuration state. The current state of the
UX FLAWS: DEFINITIONS AND CRITERIA configuration (e.g. what options were selected, how the
Based on our observations, we elaborated a list of common final product looks like) is not displayed.
design flaws typically found in configurators. We catego- Configuration. Finally, the lack of essential configurator-
rize them into three degrees of specifity. The first degree specific functionalities can lead to bad UX, as reported by [12].
comprises violations of general HCI principles that any ap- More precisely, we consider the following flaws:
plication should satisfy. The second concerns issues related
to navigation and are thus commonly found in interactive • Unchecked errors. The configurator allows the user to
applications that represent a process. The last degree is re- make incompatible choices, which lead to invalid configu-
lated to the specific functionalities found in configurators, like rations and ultimately to infeasible products.
propagations and repairs. • No explanation. The configurator does not explain why
errors are raised or why some options became mandatory
General HCI. We selected four types of violations that we or unavailable.
deem critical in configurators. We settle for those, as our aim • No repair. The configurator does not provide any repair
is not to perform a thorough analysis but rather to study how functionality that helps the user fixing an error or changing
often such criteria are unsatisfied in practice. These are: her configuration, e.g., to make available forbidden options.
• Absence of feedback. No feedback is given to the user.
There is no apparent result to her action.
• Information overload. The quantity of displayed informa- RESULTS
tion is too high, or there is too much noise. For each of the 28 configurators, we check whether its UI
• Too many controls. The UI displays too many controls on exhibits the 11 aforementioned design flaw. We first observe
a single screen, or the number of controls needed to perform the occurrence of each design flaws. Accordingly, Figure 7
a single action is too high. presents for each flaw how many configurators exhibit it.
Figure 7. Number of configurators that exhibit each flaw
For general HCI principles, we observe that the lack of er-
gonomics is the most common pitfall, with as many as 21
configurators concerned with such issues. This is not totally
surprising, as the ergonomics guidelines we consider are rather
large, although we could expect better ergonomics in such
customer-oriented applications. Information overload follows
next, with half of the configurators being problematic. It is
indeed common that car configurators have to deal with large
amounts of options, hence the difficulty of presenting all these
options while avoiding cognitive overload. 11 configurators
contain too many controls. According to our observations, this
is the result of overusing categorization to limit the number
of options displayed to the user at once. Finally, only six
configurators do not provide feedback. Overall, we notice
that six configurators out of the 28 do not violate our general
HCI principles, which may reveal an inherent difficulty to
satisfying all these principles in this kind of application.
Regarding navigation issues, seven configurator UIs do not
present clearly the navigation steps that the user has to follow. Figure 8. Number of flaw types occurring in configurators
Furthermore, the navigation controls are irregular in 14 cases,
while ten configurators do not display any progress. More
importantly, most of the configurators (23 cases) do not show
These results clearly show that even though configurators are
the current status of the configuration, which is increasingly
often infected by common UX flaws, the most important issues
important as the number of options gets higher. These nav-
are specific to the configuration problem. These include not
igation problems likely interfere with the user’s experience;
displaying the current configurator state and not providing
systematic solutions should thus be provided.
essential configuration functionalities. Moreover, even if these
The most recurring problems, however, originate from functionalities were provided, there would remain the question
configuration-specific functionalities. 23 out of 28 configura- of how to integrate them properly in a configuration UI.
tors do not systematically check the absence of errors, thereby
authorising invalid configurations. We also identified 13 cases
where explanations are not always given following an error CONCLUSION
or a propagation. Finally, 18 configurators do not provide Configurators are intelligent and highly-interactive systems
automated repair each time they should. that play a major role in today’s businesses. Despite their
importance, our study reveals major issues in regard to their
Figure 8 gives us insights into the number of flaw types that UX. On the other side, no HCI standards exist for these ap-
occur in the configurators. We observe that half of the con- plications, and only few research work were interested in
figurators violate more than six of our principles. This can configurator UX. This motivates our long-term objective to
be explained by the fact that the three configuration-specific design HCI guidelines specific to configurators, taking into
functionalities are often missing altogether, while unknown account all their functionalities and particularities. In the fu-
configuration state and ergonomic problems are also pretty ture, we plan to expand our study across multiple industries in
common. More surprisingly, only a quarter of the configura- order to generalise our conclusions. This way, we also hope
tors comprise three violations or less. In the end, only one to identify the sources of the different UX problems, thereby
configurator does not contain any flaw. These observations paving the way to our envisioned elaboration of guidelines.
tend to show the necessity of establishing guidelines to be Acknowledgement This work was partly supported by the Eu-
followed by every configurator. ropean Commission (FEDER IDEES/CO-INNOVATION) and
the Wallonia-Brussels Federation under the ARC programme.
REFERENCES 14. D. Sabin and R. Weigel. 1998. Product configuration
1. Ebrahim Khalil Abbasi, Arnaud Hubaux, Mathieu Acher, frameworks-a survey. IEEE Intelligent Systems and their
Quentin Boucher, and Patrick Heymans. 2013. The Applications 13, 4 (Jul 1998), 42–49. DOI:
Anatomy of a Sales Configurator: An Empirical Study of http://dx.doi.org/10.1109/5254.708432
111 Cases. In CAiSE’13. Springer-Verlag, Berlin,
15. Dominique L Scapin and JM Christian Bastien. 1997.
Heidelberg, 162–177.
Ergonomic criteria for evaluating the ergonomic quality
2. Liliana Ardissono, Alexander Felfernig, Gerhard of interactive systems. Behaviour & information
Friedrich, Dietmar Jannach, Ralph Schäfer, and Markus technology 16, 4-5 (1997), 220–231.
Zanker. 2002. A Framework for Rapid Development of
16. Ben Shneiderman. 1997. Designing the User Interface:
Advanced Web-based Configurator Applications. In
Strategies for Effective Human-Computer Interaction
ECAI’02. IOS Press, Amsterdam, The Netherlands, The
(3rd ed.). Addison-Wesley, Boston, MA, USA.
Netherlands, 618–622.
17. C. Streichbier, P. Blazek, and F. Faltin. 2009. Are
3. Maxime Cordy and Patrick Heymans. 2018. Engineering De-Facto Standards a Useful Guide for Designing
Configurators for the Retail Industry: Experience Report Human-Computer Interaction Processes? The Case of
and Challenges Ahead (to appear). In SAC ’18. ACM. User Interface Design for Web Based B2C Product
4. Cyledge. 2017. Configurator Database. (2017). Retrieved Configurators. In HICSS ’09. IEEE Computer Society,
July 24, 2017 from Washington, DC, USA, 1–7.
http://www.configurator-database.com
18. Jean Vanderdonckt. 1994. Guide ergonomique des
5. Alexander Felfernig, Lothar Hotz, Claire Bagley, and interfaces homme-machine. Presses Universitaires de
Juha Tiihonen. 2014. Knowledge-based Configuration: Namur (1994).
From Research to Business Cases (1 ed.). Morgan 19. J. White, D. C. Schmidt, D. Benavides, P. Trinidad, and
Kaufmann Publishers Inc., San Francisco, CA, USA. A. Ruiz-Cortés. 2008. Automated Diagnosis of
6. Gerhard Fleischanderl, Gerhard E. Friedrich, Alois Product-Line Configuration Errors in Feature Models. In
Haselböck, Herwig Schreiner, and Markus Stumptner. SPLC ’08. IEEE Computer Society, Washington, DC,
1998. Configuring Large Systems Using Generative USA, 225–234.
Constraint Satisfaction. IEEE Intelligent Systems 13, 4 20. Yingfei Xiong, Arnaud Hubaux, Steven She, and
(July 1998), 59–68. Krzysztof Czarnecki. 2012. Generating Range Fixes for
7. Lothar Hotz, Alexander Felfernig, Andreas Günter, and Software Configuration. In ICSE ’12. IEEE Press,
Juha Tiihonen. 2014. A short history of configuration Piscataway, NJ, USA, 58–68.
technologies. Knowledge-based Configuration–From
Research to Business Cases (2014), 9–19.
8. Arnaud Hubaux, Yingfei Xiong, and Krzysztof Czarnecki.
2012. A User Survey of Configuration Challenges in
Linux and eCos. In VaMoS ’12. ACM, 149–155.
9. Tony Leclercq, Jean-Marc Davril, Maxime Cordy, and
Patrick Heymans. 2016. Beyond De-Facto Standards for
Designing Human-Computer Interactions in
Configurators. In EnCHIReS@EICS 2016, Bruxelles,
Belgium. 40–43.
10. Jakob Nielsen. 2005. Ten usability heuristics. (2005).
11. B.J. Pine and S. Davis. 1999. Mass Customization: The
New Frontier in Business Competition. Harvard Business
School Press.
12. Rick Rabiser, Paul Grünbacher, and Martin Lehofer.
2012. A Qualitative Study on User Guidance Capabilities
in Product Configuration Tools. In Proceedings of the
27th IEEE/ACM International Conference on Automated
Software Engineering (ASE 2012). ACM, New York, NY,
USA, 110–119. DOI:
http://dx.doi.org/10.1145/2351676.2351693
13. RetailWeek. 2017. Personalisation. (2017). Retrieved July
24, 2017 from https://www.retail-week.com/topics/
technology/personalisation