Recommending new and relevant destinations to travellers is one of the most important use cases for Skyscanner. Internal research shows that more than 51% of Skyscanner users have not yet chosen their next destination, making them more open to recommendations. Recent advances in recommendation models for the travel industry help address the question of “what” to recommend, but various practical challenges remain for companies to eficiently serve these recommendations to users. For instance, how to deal with diverse set of product items (countries, cities, airports, etc.,), how to tackle the cold start problem, and how to increase adoption of recommendations across the organisation. In this work we describe Skyscanner's approach to solving these problems by migrating from a decentralised to a centralised destination recommendation architecture. This migration has increased the number of frontends serving recommendations, reduced redundant implementation eforts, and accelerated the experimentation pace, among other benefits.
CEUR
ceur-ws.org multiple Skyscanner frontends, how we created a centralised architecture to mitigate those challenges and what impact we have seen with the new architecture.
Recommendation models are often purpose-built for each frontend touchpoint, which is ultimately responsible for constructing the destination cards displayed to users. As a result, each frontend must separately integrate with various services—such as images, destination labels, and price services—to present enticing recommendations to travelers. This architecture is shown in Figure 1.
We realised that this architecture did not scale well when attempting to increase the adoption of destination recommendations across Skyscanner’s frontend clients. It led to duplicated eforts, as each frontend had to integrate separately with recommendation models and backend services for their specific use case. For data scientists, this architecture introduced additional challenges, such as monitoring model performance, maintaining expected model behaviour, and measuring overall impact of recommendations on the business. They had limited visibility into how frontend clients were using the recommendations (e.g. whether clients applied any post-filtering logic) and how travellers interacted with the recommendations, as each frontend could implement custom logging.
Travellers visiting Skyscanner can explore and book flights, accommodations, and rental cars. This diversity of verticals means that there are numerous frontends where destination recommendations can be presented to users. These frontends can serve diferent search intents (e.g. “exploring everywhere” vs “searching cities in a specific geographical area”) and allow travellers to express these intents in various ways. As a result, diferent recommendation models are required to recommend distinct types of destinations—i.e. countries, cities, or airports—depending on where the user is in their exploration journey. This introduces another challenge for the frontend client, which is determining the appropriate recommendation model.
Frontend clients had to be deliberate in choosing the appropriate recommendation model for their use case, and pass the required parameters to the model. This challenge was compounded by the lack of visibility into available recommendation models, as there was no centralised repository or model catalog. Each integrating client had to contact the recommendations team to enquire about available models, determine if any were suitable for their use case, and assess whether they could utilise them by providing the required search context. The recommendations team, in turn, had to handle these requests on a case-by-case basis, making it dificult to apply learnings from previous integrations.
Sparse interaction data is often a common problem for recommendation algorithms and this is
exacerbated in the travel industry due to the inherent infrequent nature of user activity on these sites
[
Skyscanner doesn’t require users to be logged in to explore destinations, hence we can’t have a long history of user interactions with destinations, because user cookies expire. Moreover, a long history of interactions with the destinations may not always be relevant, because travellers want diferent experiences and can have polar-opposite preferences from one trip to another (e.g. a safari get-away to Masai Mara in May and a business trip to London in September). This means that our recommendation platform needs to be flexible to provide recommendations for all types of users.
The more we know about the traveller’s preferences from their past interactions, the more relevant recommendations can be provided. While most of our recommendation models have built-in fallbacks for cold start cases, it is still good practice for frontend clients to implement their own fallbacks in case a model can’t generate recommendations. However, this poses the challenge of limited observability of these fallbacks being used and what recommendations they provide to the traveller. Additionally, diferent fallback logic across Skyscanner frontends can result in an inconsistent user experience for travellers navigating across our product.
The new architecture with a centralised destination recommendations service was designed and developed to address the challenges outlined in the previous section, with the primary goal of increasing internal adoption of destination recommendations across Skyscanner frontends. To achieve this, the new architecture had to meet two key requirements: • Simplify the integration processes for frontend clients wanting to use recommendations. • Relieve frontend clients from having to decide which recommendation model is the most suitable for their specific use case.
Skyscanner travellers can begin searching for their next trip from various Skyscanner frontends (e.g. main homepage, flight, hotel, or car hire search pages, or multiple landing pages ). Travellers may either search for specific destinations (e.g. “flights from London to Bari”), or explore with open-ended searches (e.g. “flights everywhere” or “flights from Bari to Spain”).
The process described in Figure 2 only applies to open-ended searches. These searches help us capture various traveller intents and require diferent types of recommendations, such as countries, cities and airports. Examples of recommendations for diferent traveller intents are shown in Figure 3.
This service is a single point of entry to Skyscanner’s search-related backend services. Its goal is to provide standardised and easy access to distinct types of information for flights, hotels, cars through a unified service API. In other words, it abstracts frontend clients from the need to directly integrate with each backend service.
When the USS receives an open-ended search, it forwards the request to the Destination Recommendations Service to obtain relevant recommendations. Once the USS receives the recommendations, it queries multiple backend services to enrich them with geographical information, prices, images and checks flights availability.
The Destination Recommendations Service acts as a routing layer with the following responsibilities: (a) “Explore Everywhere” feature on Skyscanner recommending countries to inspire travellers (b) Skyscanner landing page widget with relevant cities where travellers might want to book hotels (c) Skyscanner landing page widget with most popular airports to rent cars from Europcar • It extracts the search intent from the context of the incoming request and selects the most relevant recommendation model. • It modifies the request to retrieve recommendations from the model hosted in the Model Serving
Platform.
• It returns recommendations to the USS.
The service also includes fallbacks for each model, used in some cold start cases or if a model is unable to return recommendations. These fallbacks are sensible defaults that provide some non-refined results to the average user (e.g. most popular destinations).
This platform hosts the majority of Skyscanner’s machine learning models used for real-time inference. It also provides access to a low-latency Feature Store that can be used to enrich models at inference time with traveller preferences and destination features. For example, in item-to-item collaborative ifltering models, we can retrieve a traveller’s recent searches from the Feature Store and use this data to recommend similar destinations.
The Feature Store can be also queried directly to obtain relevant content without needing to call a model. For example, a service that constructs marketing email to re-engage users queries Feature Store directly to obtain personalised destination recommendations generated in a batch process.
Additionally, the Model Serving Platform emits logs for model requests and responses, which are used for ML observability purposes and model optimisation.
Traveller interaction data and metadata for various destination types are processed using ETL pipelines. These pipelines operate at diferent frequencies and are also used to train various recommendation models. The trained models are stored in a model registry and are made available for inference through the Model Serving Platform.
Some outputs of these ETL pipelines are also saved to the Feature Store. Examples include user features used to personalise models based on recent traveller preferences and searches, destination popularity scores that serve as model fallbacks, or destination vibes.
In this section, we present the results and business impact following the roll-out of the new architecture almost in 2023 (almost a year ago).
The new architecture removed the need for each frontend client to integrate individually with recommendation models and multiple backend services. This streamlined process reduced implementation overhead and minimised duplication of eforts, leading to more eficient resource allocation.
Frontends no longer need to determine which specific recommendation model is suitable for their use case. The Destination Recommendations Service now handles this by interpreting the request and routing it to the most relevant model.
Since the Destination Recommendations Service includes fallbacks, frontend clients no longer need to create their own solutions for cold start cases or when a model returns no results. Centralising the fallbacks ensures the quality of the single source of truth and maintains consistency of user experience across diferent frontends.
The new architecture has improved the observability of model performance across multiple clients, leading to better standardised monitoring and enhancements in recommendation quality. It has also standardised alerting and accelerated incident response times.
We have implemented Service Level Objectives (SLOs) for all integrating clients, with the service now expected to respond to all incoming requests within 100 milliseconds. These improvements significantly improved the overall experience for integrating clients.
The rate of experimentation also notably increased due to the streamlined integration and decisionmaking processes, as well as the accelerated model development enabled by the new architecture. For data scientists, this enhanced observability of user interactions with recommended destinations across Skyscanner frontends has ofered valuable insights into the performance of recommendation models in diferent contexts, enabling more efective decision-making regarding model retraining and maintenance.
The simplified integration and all the added benefits of the new architecture have led to a significant increase in the use of destination recommendations across Skyscanner.
Prior to the development of the new architecture, we had 3 frontends using recommendation models in their exploration flows. Potential frontend clients, who wanted to leverage recommendations in their touchpoint, would always highlight the extra efort and complexity of integrating with multiple backend services to create the destination cards as a blocker. However, since the roll-out of the centralised service to clients in May 2023, we have seen a steady increase in the number of frontend clients that have integrated due to the simplified process.
Currently, destination recommendation models are used by 8 Skyscanner frontends and handle 960 million recommendation requests to 110 million Skyscanner users from 180 countries every month1. This represents a significant increase from the approximately 160 million monthly recommendation requests and 20 million users we had at the start of 2023 (Figure 4).
Given the high proportion of users who visit Skyscanner to explore and plan new trips, recommending travel destinations is one of the platform’s main oferings. Providing item recommendations has proven valuable across various fields, and the travel sector is no exception. This has led to a wide range of available state-of-the-art recommendation models that are ready to be trained and tested. However, the challenges that companies face when implementing recommender systems are often overlooked. These challenges include ensuring high internal adoption by diferent frontend teams, improving the developer experience during integration, dealing with the cold start problem, and interpreting traveller search intent and matching it with relevant recommendations from a diverse catalogue. In this paper, 1Average values across June & July 2024 we have discussed Skyscanner’s journey of implementing a centralised destination recommendation system and how it has efectively addressed these challenges.
Future work will focus on further improvements to the recommendation models, exploring more advanced machine learning techniques, and expanding recommendations into new areas of the business. This work on the new architecture with centralised recommendations service has given us a template for potentially centralising architecture for other types of recommendations as well, such as e.g. hotel recommendations that have a very diferent stack of backend services and have their own set of challenges.