On Studying Bad Practices in Configuration UIs Tony Leclercq, Maxime Cordy, Bruno Dumas, Patrick Heymans University of Namur, Belgium first.last@unamur.be ABSTRACT In today’s businesses, configurators are essential tools that al- low customers to personalise a product to their specific needs. Being often the first contact between the retailer and the cus- tomer, it is important for a configurator to provide a smooth user experience. However, these software have been studied mostly from the reasoning point of view, that is, how to rep- resent the logical relations between the configuration options and how to automatically ensure that the user makes no con- figuration error. In this paper, we focus instead on HCI and analyse the occurrence of 11 design flaws that occur in 28 car configurators. These flaws includes violations of general HCI principles as well as lacks of essential, configurator-specific Figure 1. Some car configurator functionalities. Our results indicate that most of the studied configurators contain defects in their UIs including, surpris- ingly, violations of some basic principles of HCIs. Still, the car configurator where customers can choose between various most frequent problems are inappropriate implementations of options like trim, motor, colour, accessories, etc. Beneath the configurator-specific functionalities. user interface lies an intelligent system that provides effec- tive guidance by preventing errors and providing automated ACM Classification Keywords support for the configuration task [5, 8]. An error is typically H.5.2. User Interfaces: Graphical user interfaces (GUI), Con- an attempt to include incompatible options or parameter val- figurator, Evaluation, Intelligent system ues in the configuration. When the user makes an error, the configurator should alert her and explain why the configu- INTRODUCTION ration is invalid. On the other side, when previous choices The avenue of mass customisation – an established marketing make mandatory to select or to exclude an option, or when and engineering paradigm that unifies mass production and a parameter accepts only one value, the configurator should customisation [11] – has yielded new habits in customers’ propagate (i.e., force) this choice so as to simplify the task of behaviour. In constant search of the ideal product (i.e. good or the user. For instance, selecting a comfort trim automatically service) that satisfies their particular needs, they hunger for the includes a cruise control system; see Figure 2). It is again freedom of tailoring sellers’ catalog into a personalised offer. recommended to explain the cause of the propagation, in case It is therefore not surprising that personalisation is considered the user disagrees with this choice. When errors or undesired as one of the hottest topics in the retail industry [13]. Still, to propagations occur, the configurator should help repair the be successful personnalisation must be supported by a smooth configuration [19, 20], that is, automatically compute what user experience (UX) that offers the highest customisation must be changed to fix the error or undo the propagation, re- capability while hiding the inherent complexity [12]. spectively. An example of repair system is shown in Figure 3. Configurator is the typical software support for mass customi- We see that the selected wheels require a particular suspension sation. It consists in an interactive application where the user system and specific tires, as well as removing the off-road specifies its requirements by selecting options and setting pa- accessory pack. rameter values. As a result she obtains the relevant products Configuration knowledge refers to the set of rules that deter- that can address these needs. For example, Figure 1 shows a mine what is a valid configuration. It originates from rational (technical, legal, mathematical) and subjective (marketing, aes- thetics) knowledge. The acquisition of this knowledge is a critical analysis activity for a successful configurator [2, 6], as erroneous or incomplete knowledge can have drastic conse- quences on the products that will be built afterward. Once the configuration knowledge is defined, one can develop the back- ©2018. Copyright for the individual papers remains with the authors. end part of the configurator responsible for checking errors, as Copying permitted for private and academic purposes. well as computing propagations and repairs. A standard way WII’18, March 11, 2018, Tokyo, Japan Figure 4. Violations of basic ergonomic guidelines: empty controls, Figure 2. Propagation of mandatory option empty boxes and wrong control placements commonly provide a high number of options (which makes both presentation and reasoning challenging), they are directly aimed at customers (which makes UX a primary concern), and we can find very different UI design choices across different brands and countries. More precisely, we check whether 11 UX design flaws occur in 28 configurators we selected across different brands and countries. These 11 flaws comprise 4 general HMI anti-patterns, 4 specific to navigation-based in- terfaces, and 3 related to configurator-specific functionalities. Figure 3. An instance of repair where some options must be added and Our results reveal that 75% of the configurators show more others removed than three types of UX flaws, half of them more than six. This is mostly due to missing or badly implemented configuration- specific functionalities, although we also observe the violation to do that is to implement it from scratch in an established of general HCI principles. This corroborates the need for HCI programming language like Java or PHP. However, this results guidelines specific to configurator engineering. in scattering the knowledge across the code base, which leads to increased risks of bugs and poor maintainability [1]. An AN OVERVIEW OF UX ISSUES IN CONFIGURATORS alternative solution is to encode it as logical rules in a configu- To carry out our study, we select a set of car configurators ration engine [14, 7, 3], i.e., a dedicated software component amongst the ten top-selling car manufacturers. For each man- that implements generic algorithms to reason on configura- ufacturer we choose the best selling car model, and for every tion knowledge. Such engines thus provide the configuration model we consider the configurator of the US, UK and Bel- functionality at no cost and regardless of the actual application gium websites. This leads us to 28 configurators to evaluate domain. (as two of the 30 candidates do not allow one to select any option). While efficient configuration engines exist, they are only a part of the challenges in configurator engineering. Indeed, configu- Looking at these configurators, we rapidly note that basic er- rators belong to a class of interactive applications where UX gonomic guidelines [18, 15, 16, 10] are not satisfied. Indeed, is of paramount importance [16] and relies on specific func- we observe occurrences of empty controls, empty boxes or tionalities of these intelligent systems [12]. However, there wrong control placements; see Figure 4. Inappropriate choice exist neither established nor de-facto HMI standards for con- of controls, inconsistent semantics grouping, and lack of em- figurators [17, 9]. Even worse, recent studies [1] showed that phasis were also very common. This is relatively surprising, many public configurators do not follow general HMI guide- as car configurators are popular and powerful marketing tools. lines, resulting in weak usability. This reveals that designing This makes us consider general HCI principles as our first HMI guidelines for configurators is a essential element of the evaluation criteria. Our objective being more focused than a envisioned “body of knowledge dedicated to the engineering general-purpose HCI evaluation, we settle for only four viola- of configurators” [1, 3]. tions we observed the most: absence of feedback, information overload, too many controls and bad ergonomics (see more in A prerequisite to creating these guidelines is to understand the next section). what existing configurators do wrong. Therefore, pursuing previous research [12, 1], in this paper we analyse to what In addition to these general considerations, we also observe extent existing configurators contain HCI design flaws. We issues related to the management of navigation within the focus more particularly on car configurators, the most popu- UI. Figure 5 is a striking example that highlights multiple lar kind of configurators according to the Cyledge’s database problems. First, we see that the interface does not display the statistics [4]. Car configurators are excellent candidates: they step currently completed. Also it provides no visible way to Figure 5. All signs of navigation have disappeared Figure 6. Invalid configuration: multiple wheel sizes are selected go to the previous or the next step. The user has no idea of the progress she has already accomplished, and can neither • Bad ergonomics. The configurator does not follow estab- consult nor preview the current state of her configuration. All lished ergonomics guidelines. These include, e.g., mis- these navigation-related issues constitute our second group of placed controls and inappropriate colour or font size. criteria. Navigation. Configuration being an interactive process, its Finally, we observe the lack of configuration-specific func- navigation must be absolutely clear. We can see navigation tionalities that are, however, deemed essential. For instance, from three points of view, i.e. in the UI, in the progress in the Figure 6 depicts a configuration where three different sizes of process, and in the configuration state. This leads us to four wheels are selected. Errors are thus unchecked in this configu- types of violations: rator. Configuration being a long and error-prone process, it is unrealistic not to help users complete their task. Explana- • Unknown UI location. The user is lost within the UI and tion of errors, propagations and repairs – as presented in the has no idea what step she is currently completing. previous section – are other facilitating functionalities. We • Irregular navigation control. In the UI, the user cannot argue that all configurators should implement them. Together see the control allowing her to move to the next step, be- with error checking, they are thus part of our third type of cause their position or shape change. evaluation criteria. All our criteria are properly defined in the • Unknown progress. The progress in the process, i.e. what next section. percentage of effort the user has already put in or remains to be done, is not displayed. • Unknown configuration state. The current state of the UX FLAWS: DEFINITIONS AND CRITERIA configuration (e.g. what options were selected, how the Based on our observations, we elaborated a list of common final product looks like) is not displayed. design flaws typically found in configurators. We catego- Configuration. Finally, the lack of essential configurator- rize them into three degrees of specifity. The first degree specific functionalities can lead to bad UX, as reported by [12]. comprises violations of general HCI principles that any ap- More precisely, we consider the following flaws: plication should satisfy. The second concerns issues related to navigation and are thus commonly found in interactive • Unchecked errors. The configurator allows the user to applications that represent a process. The last degree is re- make incompatible choices, which lead to invalid configu- lated to the specific functionalities found in configurators, like rations and ultimately to infeasible products. propagations and repairs. • No explanation. The configurator does not explain why errors are raised or why some options became mandatory General HCI. We selected four types of violations that we or unavailable. deem critical in configurators. We settle for those, as our aim • No repair. The configurator does not provide any repair is not to perform a thorough analysis but rather to study how functionality that helps the user fixing an error or changing often such criteria are unsatisfied in practice. These are: her configuration, e.g., to make available forbidden options. • Absence of feedback. No feedback is given to the user. There is no apparent result to her action. • Information overload. The quantity of displayed informa- RESULTS tion is too high, or there is too much noise. For each of the 28 configurators, we check whether its UI • Too many controls. The UI displays too many controls on exhibits the 11 aforementioned design flaw. We first observe a single screen, or the number of controls needed to perform the occurrence of each design flaws. Accordingly, Figure 7 a single action is too high. presents for each flaw how many configurators exhibit it. Figure 7. Number of configurators that exhibit each flaw For general HCI principles, we observe that the lack of er- gonomics is the most common pitfall, with as many as 21 configurators concerned with such issues. This is not totally surprising, as the ergonomics guidelines we consider are rather large, although we could expect better ergonomics in such customer-oriented applications. Information overload follows next, with half of the configurators being problematic. It is indeed common that car configurators have to deal with large amounts of options, hence the difficulty of presenting all these options while avoiding cognitive overload. 11 configurators contain too many controls. According to our observations, this is the result of overusing categorization to limit the number of options displayed to the user at once. Finally, only six configurators do not provide feedback. Overall, we notice that six configurators out of the 28 do not violate our general HCI principles, which may reveal an inherent difficulty to satisfying all these principles in this kind of application. Regarding navigation issues, seven configurator UIs do not present clearly the navigation steps that the user has to follow. Figure 8. Number of flaw types occurring in configurators Furthermore, the navigation controls are irregular in 14 cases, while ten configurators do not display any progress. More importantly, most of the configurators (23 cases) do not show These results clearly show that even though configurators are the current status of the configuration, which is increasingly often infected by common UX flaws, the most important issues important as the number of options gets higher. These nav- are specific to the configuration problem. These include not igation problems likely interfere with the user’s experience; displaying the current configurator state and not providing systematic solutions should thus be provided. essential configuration functionalities. Moreover, even if these The most recurring problems, however, originate from functionalities were provided, there would remain the question configuration-specific functionalities. 23 out of 28 configura- of how to integrate them properly in a configuration UI. tors do not systematically check the absence of errors, thereby authorising invalid configurations. We also identified 13 cases where explanations are not always given following an error CONCLUSION or a propagation. Finally, 18 configurators do not provide Configurators are intelligent and highly-interactive systems automated repair each time they should. that play a major role in today’s businesses. Despite their importance, our study reveals major issues in regard to their Figure 8 gives us insights into the number of flaw types that UX. On the other side, no HCI standards exist for these ap- occur in the configurators. We observe that half of the con- plications, and only few research work were interested in figurators violate more than six of our principles. This can configurator UX. This motivates our long-term objective to be explained by the fact that the three configuration-specific design HCI guidelines specific to configurators, taking into functionalities are often missing altogether, while unknown account all their functionalities and particularities. In the fu- configuration state and ergonomic problems are also pretty ture, we plan to expand our study across multiple industries in common. More surprisingly, only a quarter of the configura- order to generalise our conclusions. This way, we also hope tors comprise three violations or less. In the end, only one to identify the sources of the different UX problems, thereby configurator does not contain any flaw. These observations paving the way to our envisioned elaboration of guidelines. tend to show the necessity of establishing guidelines to be Acknowledgement This work was partly supported by the Eu- followed by every configurator. ropean Commission (FEDER IDEES/CO-INNOVATION) and the Wallonia-Brussels Federation under the ARC programme. REFERENCES 14. D. Sabin and R. Weigel. 1998. Product configuration 1. Ebrahim Khalil Abbasi, Arnaud Hubaux, Mathieu Acher, frameworks-a survey. IEEE Intelligent Systems and their Quentin Boucher, and Patrick Heymans. 2013. The Applications 13, 4 (Jul 1998), 42–49. DOI: Anatomy of a Sales Configurator: An Empirical Study of http://dx.doi.org/10.1109/5254.708432 111 Cases. In CAiSE’13. Springer-Verlag, Berlin, 15. Dominique L Scapin and JM Christian Bastien. 1997. Heidelberg, 162–177. Ergonomic criteria for evaluating the ergonomic quality 2. Liliana Ardissono, Alexander Felfernig, Gerhard of interactive systems. Behaviour & information Friedrich, Dietmar Jannach, Ralph Schäfer, and Markus technology 16, 4-5 (1997), 220–231. Zanker. 2002. A Framework for Rapid Development of 16. Ben Shneiderman. 1997. Designing the User Interface: Advanced Web-based Configurator Applications. In Strategies for Effective Human-Computer Interaction ECAI’02. IOS Press, Amsterdam, The Netherlands, The (3rd ed.). Addison-Wesley, Boston, MA, USA. Netherlands, 618–622. 17. C. Streichbier, P. Blazek, and F. Faltin. 2009. Are 3. Maxime Cordy and Patrick Heymans. 2018. Engineering De-Facto Standards a Useful Guide for Designing Configurators for the Retail Industry: Experience Report Human-Computer Interaction Processes? The Case of and Challenges Ahead (to appear). In SAC ’18. ACM. User Interface Design for Web Based B2C Product 4. Cyledge. 2017. Configurator Database. (2017). Retrieved Configurators. In HICSS ’09. IEEE Computer Society, July 24, 2017 from Washington, DC, USA, 1–7. http://www.configurator-database.com 18. Jean Vanderdonckt. 1994. Guide ergonomique des 5. Alexander Felfernig, Lothar Hotz, Claire Bagley, and interfaces homme-machine. Presses Universitaires de Juha Tiihonen. 2014. Knowledge-based Configuration: Namur (1994). From Research to Business Cases (1 ed.). Morgan 19. J. White, D. C. Schmidt, D. Benavides, P. Trinidad, and Kaufmann Publishers Inc., San Francisco, CA, USA. A. Ruiz-Cortés. 2008. Automated Diagnosis of 6. Gerhard Fleischanderl, Gerhard E. Friedrich, Alois Product-Line Configuration Errors in Feature Models. In Haselböck, Herwig Schreiner, and Markus Stumptner. SPLC ’08. IEEE Computer Society, Washington, DC, 1998. Configuring Large Systems Using Generative USA, 225–234. Constraint Satisfaction. IEEE Intelligent Systems 13, 4 20. Yingfei Xiong, Arnaud Hubaux, Steven She, and (July 1998), 59–68. Krzysztof Czarnecki. 2012. Generating Range Fixes for 7. Lothar Hotz, Alexander Felfernig, Andreas Günter, and Software Configuration. In ICSE ’12. IEEE Press, Juha Tiihonen. 2014. A short history of configuration Piscataway, NJ, USA, 58–68. technologies. Knowledge-based Configuration–From Research to Business Cases (2014), 9–19. 8. Arnaud Hubaux, Yingfei Xiong, and Krzysztof Czarnecki. 2012. A User Survey of Configuration Challenges in Linux and eCos. In VaMoS ’12. ACM, 149–155. 9. Tony Leclercq, Jean-Marc Davril, Maxime Cordy, and Patrick Heymans. 2016. Beyond De-Facto Standards for Designing Human-Computer Interactions in Configurators. In EnCHIReS@EICS 2016, Bruxelles, Belgium. 40–43. 10. Jakob Nielsen. 2005. Ten usability heuristics. (2005). 11. B.J. Pine and S. Davis. 1999. Mass Customization: The New Frontier in Business Competition. Harvard Business School Press. 12. Rick Rabiser, Paul Grünbacher, and Martin Lehofer. 2012. A Qualitative Study on User Guidance Capabilities in Product Configuration Tools. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE 2012). ACM, New York, NY, USA, 110–119. DOI: http://dx.doi.org/10.1145/2351676.2351693 13. RetailWeek. 2017. Personalisation. (2017). Retrieved July 24, 2017 from https://www.retail-week.com/topics/ technology/personalisation