A Framework for Comparing Interactive Route Planning Apps in Tourism Sergejs Pugacs, Sven Helmer, Markus Zanker Free University of Bozen-Bolzano Bolzano, Italy Sergejs.Pugacs@stud-inf.unibz.it, shelmer@inf.unibz.it, mzanker@unibz.it Abstract. Route planning applications are digital companions of many travelers, enabling them to experience different locations in a self-determined way. This work surveys the variety of commercial applications and re- search prototypes for itinerary planning with the goal of evaluating the interactivity aspect of these tools. For this purpose a framework to clas- sify the interaction mechanisms and usage features of route planning applications has been developed that differentiates between the point-of- interest and the aggregate tour layer. This proposed framework is applied to classify existing apps and points to opportunities for further research. 1 Introduction Instead of relying on printed guide books and preplanned tours, more and more tourists are switching to applications to help them plan trips in a more self- determined and interactive way [6]. While there is a substantial body of work on the efficient route generation part of tourist trip planning [4], especially in the area of orienteering, we believe that the usability and interactivity of a system is a more crucial aspect. Interactivity plays a key role when it comes to user acceptance: users who have the feeling that they actively contributed to creating the trips, rather than just following the instructions of the application, will feel a sense of achievement and keep using this system. Our goal is to compare different route planning applications in terms of their level of interactivity. In order to be able to do so, we develop a comparison framework custom-tailored to the specific aspects of route planning. In particular we make the following contributions: – we investigate a set of guidelines developed for the usability of recommender systems in general and use it as a basis for our framework to evaluate the interactivity of route planning systems; – we apply our framework to state-of-the-art route planning applications, eval- uating the support of interactivity according to our defined framework. A Framework for Comparing Interactive Route Planning Apps in Tourism 19 2 Related Work In their survey Vansteenwegen and Souffriau [11] provide a detailed overview of the functionalities and features that users expect from a trip planning app. They range from personal interest estimation, automatic route generation, mandatory POIs, dynamic recalculation, multiple day decision support, opening hours, bud- get limitations, maximal type constraints, mandatory type constraints, weather dependency, scenic routes, hotel selection, public transportation, all the way to group profiles. While this is a comprehensive list, there is no clear description of how these preferences are actually elicited from users. By looking at the sheer number of the potential constraints listed above, it becomes clear that users could quickly be overwhelmed by having to supply all their preferences regarding these aspects to an app. The effort required for expressing preferences has to be balanced with the quality of a tour recommen- dation: it may be more effective to ask fewer but more specific questions in order to provide a better overall user experience. On top of this, a user should be able to state their preferences as intuitively as possible. Pu et al. [9] have investigated decision support studies in recommender sys- tems in the context of usability and interactivity. They came up with a set of eleven guidelines or best practices for designing the preference elicitation process in recommender systems. Pu et al. divided the guidelines into three categories: initial preference elicitation, preference revision, and presentation of results which can in turn be assessed based on accuracy, confidence, and effort (ACE). However, the guidelines refer to recommender systems in general and lack some specific aspects of trip planners that represent composite recommen- dations. Thus, the quality of a tour recommendation has to be judged on two distinct levels. On the one hand, we want to make sure that each individual POI (i.e. on the item level) clearly fits a user’s preference. On the other hand, there are aspects relating to the overall composition of a tour. Even if a tour contains only high-quality POIs, a user could still be unhappy, because the POIs are ordered in a suboptimal way or might not represent a good balance, leading to long traveling times or sensory overload. 3 Framework Our goal is to adapt the guidelines in [9] to interactive tour planning by building a framework for comparing the user interaction of different trip planning systems. In a first step, we define the two-level distinction into tour-related and POI- related preferences in more detail. On the tour level, we have criteria such as the total time and money spent on a trip and the preferred modes of transportation. On the POI level, a user might want to specify mandatory POIs or veto certain POIs because they have already visited them or dislike certain features (e.g. long waiting times). Some preferences may even be found on both levels: we have time constraints for the overall tour, but there may also be temporal restrictions for visiting individual POIs. 20 S. Pugacs et al. We focus on the most important functionalities identified by Vansteenwegen and Souffriau [11], disregarding advanced criteria, such as weather conditions, hotel selection, and group profiles for the moment. Table 1 summarizes the pref- erences we investigate and maps them to both levels (POIs and composite tours). Table 1. Tourist preferences for a trip Preference POI Tour Time Constraints X X Categories X Mandatory POIs X Budget X Transportation X Taking the guidelines by Pu et al. [9] as a starting point, we define seven criteria for preference elicitation in interactive tour planning, which are discussed in the following. 3.1 C1 – Flexible Expression of Preferences The first three criteria in the guidelines by Pu et al. [9] are on avoiding very rigid schemes for preference elicitation, e.g. that a user has to enter all preferences in a strict order before getting any feedback. A user should be allowed to enter preferences on any attribute they choose and in any order. Different users also have different levels of expertise and fluency, so an incremental process adapted to their knowledge and experience would be appropriate. For trip planning we therefore have to consider the following. – POI-Level Preferences: mandatory POIs and category preferences can be incrementally elicited instead of requiring all of them up-front. – Tour-Level Preferences: up-front the system does not require any tour- related preferences apart from the destination itself. 3.2 C2 – Example-Based Preference Elicitation The next criterion is about using examples during preference elicitation. This helps novice users to gain fluency in expressing preferences or stimulates users who are still uncertain about them. – POI-Level Preferences: the system should suggest potential POIs to in- clude in the tour. – Tour-Level Preferences: the system should show several examples of pre- built tours. A Framework for Comparing Interactive Route Planning Apps in Tourism 21 3.3 C3 – Preference Lookahead The criterion C3 is about going beyond the current state of expressed prefer- ences and proposes to users possible additional preferences that would extend a tour. This can also be mapped to one criterion in the guidelines by [9], allowing suggestions to the user that may not be optimal yet, but will very likely become so when adding more preferences. – POI-Level Preferences: consider showing the user POIs from categories the user did not consider to evaluate. – Tour-Level Preferences: suggest a sequence of POIs that represents a partial tour, to which a user could add more POIs to complete it. 3.4 C4 – Conflict resolution If the preferences of a user are contradictory, it may not be possible to come up with a recommendation that satisfies all the constraints. In this case the system should clearly explain how the conflicts can be resolved and what compromises would be entailed. – POI-Level Preferences: when displaying/recommending a POI show how this POI matches and/or violates specific constraints. – Tour-Level Preferences: in case the user has expressed preferences for an infeasible tour, still give him a tour suggestion with an explanation how this suggestion violates their preferences. 3.5 C5 – Trade-off Transparency In case of conflicting preferences the system needs to be transparent about them, for instance, clearly explain a trade-off between quality and costs to the user. – POI-Level Preferences: show budget, quality trade-offs between different POIs. – Tour-Level Preferences: show how different tours share the same POIs and which POIs are different between them. Show differences for transporta- tion, budget, and categories. 3.6 C6 – Result Presentation A user should not be overwhelmed by the amount of information displayed on a single page (this is especially true for small hand-held devices) and items should be displayed and ranked in a natural order. – POI-Level Preferences: for mobile displays show only a few individual POIs, but for desktop clients show a large ranked list of POIs. – Tour-Level Preferences: for mobile displays show only one tour at a time as a recommendation, but for desktop clients show several tour recommen- dations at the same time. 22 S. Pugacs et al. 3.7 C7 – Explanations The final criterion is about providing information on how the system derived a result [3]. – POI-Level Preferences: explain how the POIs match preferences. Show POI scores and highlight mandatory POIs. – Tour-Level Preferences: explain how the tour matches preferences. 4 Evaluation Based on our conceptual framework to classify the interactivity of tour planning systems we assessed existing systems in order to identify opportunities for future research and development. To be objective in our existing system review process we formulated a search query (see Listing 1.1) that we evaluated over the Scopus database 1 . We reviewed the first 200 highest cited papers for inclusion in our work. The inclusion criteria were the following: 1) the paper should describe an itinerary planning system; 2) the paper should have a description of the user interface of this system. Listing 1.1. Scopus query string (TITLE−ABS−KEY( ( i t i n e r a r i e s OR r o u t e s OR t o u r s ) AND ( p e r s o n a l i s e d OR recommendations OR p l a n n i n g ) AND ( t o u r i s m OR t o u r i s t OR t r a v e l ) ) ) AND ( LIMIT−TO ( SUBJAREA, ”COMP ” ) OR LIMIT−TO ( SUBJAREA, ” DECI ” ) OR LIMIT−TO ( SUBJAREA, ” BUSI ” ) Only 6 papers were selected by these criteria. In addition we included one more paper [7] that was not in the Scopus database, that we feel is essential in the field of interactive tour planning. In addition to the academic systems we also wanted to include commercial systems in our evaluation. To select the commercial systems we used a single Google search with the query string tourism itinerary planner and consid- ering the first 20 results. We included those results that pointed to a system which allowed a user to automatically generate a travel itinerary. Five such sys- tems were selected by us. In total 12 systems were included in our evaluation. The results of our evaluation are presented in Table 2 where we indicate for each system the support of the different categories of interactive trip planning according to our framework. The rows of the table correspond to the evaluated systems and the columns to the categories of our framework. For every criterion we evaluate systems on both the POI level (P) and tour level (T). 1 https://www.scopus.com A Framework for Comparing Interactive Route Planning Apps in Tourism 23 Table 2. Categorization of existing systems based on our framework. C1 C2 C3 C4 C5 C6 C7 Authors P T P T P T P T P TP T P T CTP - - + - - + - - - - - - - - [12] Intrigue - + + - + - + - - - + - + + [2] Tainan city - - - - - - + - - - - - - - [8] CT-Planner4 + + + - - - + - - - + - + + [7] Here to there - - - - - - - - - - - - - - [1] Travel Ontology - - - - - - - - - - - - - + [5] Interactive Design - - - - - - - - + - - + - + [10] Triphobo + + + + - - - - - - + - - - https://www.triphobo.com Inspirock + + + - + - + + - - + - - - https://www.inspirock.com RoutePerfect + + + + - - + + - - + - - + https://www.routeperfect.com Google Trips + + + + - - - - - - + + - - https://get.google.com/trips Sygic Travel + + + + + - - + - - + - - - https://travel.sygic.com/ 5 Discussion After reviewing these 12 systems we conclude that a majority of them follows an identical planning strategy. First, a user is faced with the initial query where they must specify the destination city/region/country and optionally indicate the travel dates or travel category preferences. They are then presented with an itinerary plan that can be modified manually. Modifications can happen in two ways: users remove a POI from the itinerary or insert an additional new one. When inserting a new POIs the user is supported with a large list of POIs with pictures and textual descriptions as well as potential category classifications. Two systems out of 12 follow a different planning strategy, namely Route- perfect and CT-Planner. In these systems users can interactively indicate their preference degrees for different categories and the system would automatically recompute the tour based on these preferences. For an example see Figure 1. This alternative approach allows us to achieve two things: it provides an interactive environment where the user can explore many possible itineraries with little effort. It provides an explanation of the properties of the tour, since the tour is automatically generated based on preferences indicated by the user Another alternative is offered by Inspirock’s automatic tour rescheduling. When a user tries to add a POI causing a conflict, the system promptly suggests to reschedule the tour. For those criteria that are barely or not supported at all, we sketch exemplary user stories in the following. Criterion C3 – Preference Lookahead: we think it should be possible for a system to recommend whole sub-tours for inclusion into a tour. These sub-tours could be created either by domain experts or by mining previous interaction histories. Criterion C5 – Trade-off Transparency: on the POI level this would mean not only displaying POIs, but also pointing out the implications of adding a POI to a 24 S. Pugacs et al. Fig. 1. Tour construction with Routeperfect tour, e.g., longer visiting times and higher attractiveness. On the tour level, this would involve presenting a user with several alternative routes and highlighting their qualitative and quantitative differences. Criterion C6 – Result Presentation: we hypothesize that systems could use feedback approaches enabling a user to critique specific criteria such as the length of a tour, the means of transportation, and the number of breaks. Category C7 – Explanations: we believe this criterion to be one of the most important aspects of the interaction between users and the system. It is much easier for a user to compare two options if they are explained in terms of user preferences. 6 Conclusions In this paper we defined a framework for evaluating different interactivity aspects of tour planning systems that is based on existing guidelines for recommender systems in general. However, given the dual nature of preferences in the trip planning scenarios, we need to distinguish between the POI level and the com- posite tour level. The contribution of this research lies not only in the framework but also in its exemplary evaluation by classifying existing tour planning systems in order to demonstrate potential opportunities for future work. In particular we identified trade-off analysis and tour explanations as unex- plored areas and their potential in improving the user experience in the area of tour planning. A Framework for Comparing Interactive Route Planning Apps in Tourism 25 References 1. J. L. Ambite, G. Barish, C. A. Knoblock, M. Muslea, J. Oh, and S. Minton. Get- ting from here to there: Interactive planning and agent execution for optimizing travel. In Proceedings of the 14th Conference on Innovative Applications of Arti- ficial Intelligence - Volume 1, IAAI’02, pages 862–869. AAAI Press, 2002. 2. L. Ardissono, A. Goy, G. Petrone, M. Segnan, and P. Torasso. Intrigue: Person- alized recommendation of tourist attractions for desktop and hand held devices. Applied Artificial Intelligence, 17(8-9):687–714, 2003. 3. G. Friedrich and M. Zanker. A taxonomy for generating explanations in recom- mender systems. AI Magazine, 3(3):90–98, 2011. 4. A. Gunawan, H. C. Lau, and P. Vansteenwegen. Orienteering problem: A sur- vey of recent variants, solution approaches and applications. European Journal of Operational Research, 255(2):315–332, 2016. 5. M. Houda, M. Khemaja, K. Oliveira, and M. Abed. A public transportation on- tology to support user travel planning. In 2010 Fourth International Conference on Research Challenges in Information Science (RCIS), pages 127–136, May 2010. 6. K. F. Hyde and R. Lawson. The nature of independent travel. Journal of Travel Research, 42(1):13–23, 2003. 7. Y. Kurata and T. Hara. CT-Planner4: Toward a More User-Friendly Interactive Day-Tour Planner, pages 73–86. Springer International Publishing, Cham, 2013. 8. C.-S. Lee, Y.-C. Chang, and M.-H. Wang. Ontological recommendation multi- agent for tainan city travel. Expert Systems with Applications, 36(3, Part 2):6740 – 6753, 2009. 9. P. Pu, B. Faltings, L. Chen, J. Zhang, and P. Viappiani. Usability Guidelines for Product Recommenders Based on Example Critiquing Research, pages 511–545. Springer US, Boston, MA, 2011. 10. B. Rodrı́guez, J. Molina, F. Pérez, and R. Caballero. Interactive design of person- alised tourism routes. Tourism Management, 33(4):926 – 940, 2012. 11. P. Vansteenwegen and W. Souffriau. Trip planning functionalities: State of the art and future. Information Technology & Tourism, 12(4):305–315, 2010. 12. P. Vansteenwegen, W. Souffriau, G. V. Berghe, and D. V. Oudheusden. The city trip planner: An expert system for tourists. Expert Systems with Applications, 38(6):6540 – 6546, 2011.