This paper provides a comprehensive review of the lifecycle of promotional campaigns within online travel platforms (OTPs) based on real world experience from Booking.com. It emphasizes the critical aspects that must be addressed to define, optimize and monitor successfully these promotions. Initially, we present an overview of promotions in OTPs, highlighting their unique characteristics in comparison to other industries. We continue by identifying the different aspects of a promotional campaign with focus on those that can be tuned for campaign optimization. Following this, we propose best practices for experimentation, evaluation and steering to make sure that running promotional campaigns stay compliant with the business requirements.
Promotions and Discounts are popular tools that companies use to achieve their business objectives [1]. Their prevalence has grown with the advent of e-commerce platforms, owing to the relative ease and cost-effectiveness of implementation [2,3,4]. From a business perspective, promotional campaigns can be conceptualized as strategical and tactical investments, typically involving budgeted monetary resources, aimed at attaining specific business goals such as increased sales, increased revenue, higher customer satisfaction and more.
Promotions in online travel platforms exhibit distinctive characteristics that are shaped by the unique dynamics of the tourism industry. Some of these characteristics are listed below together with their implications for the promotional campaigns • Seasonality: Tourism is inherently seasonal, characterized by distinct periods of high and low demand that drive sharp changes in prices and customer elasticity. This means that promotional campaigns may have considerable changes in business objectives and effectiveness depending on the time of the year • Connected verticals: Modern OTPs offer a wide variety of travel products, spanning from accommodations, car rentals, taxis, attractions and more. All these products conform
RecTour '24 -ACM RecSys Workshop on Recommenders in Tourism, October 18, 2024, Bari, Italy Envelope carlos.herrero@booking.com (C. Herrero-Gómez); amit.livne@booking.com (A. Livne); itsik.adiv@booking.com (I. Adiv); hugo.proenca@booking.com (H. M. Proença); felipe.moraes@booking.com (F. Moraes); javier.albert@booking.com (J. Albert); dima.goldenberg@booking.com (D. Goldenberg) to a complex ecosystem in which an intervention in any of them-such as a promotioncan affect the rest. These effects can be either positive synergies but also unwanted negative results such as cannibalisation. Proper promotion management should be aware of these effects and take actions to maximize positive outcomes and mitigate the negative ones. • Shared supply: Travel products are not only limited but also often listed on several competing OTPs and on the business itself. Promotions can play a role in differentiating the value provided to the customer by offering perks like Free Cancellation, Free Breakfast or Room Upgrade. • Promotional costs: In funnel OTP's promotional campaigns only incur costs when the customer materializes the given promotion by making a purchase. For example a 10 € discount for a hotel room will only incur in a cost if the customer books that room. Unlike other promotional channels such as Pay Per Click or marketing. • Low interaction frequency: as a reflection of the tourism seasonality, customers don't engage with OTPs on a frequent basis [5]. This is in contrast to other industries such as online retail or streaming where the interactions are more recurrent. This limited and sporadic amount of interactions plays a role in campaign personalisation efforts, as described in section 2. • International nature: A big proportion of travel products and their associated promotional campaigns frequently span multiple countries, e.g. a family from the United States traveling to Europe. This cross-border nature implies compliance with the regulations of both the origin and destination countries, increasing the complexity of campaign management. Furthermore, the geo-cardinality of potential market segmentation in the travel industry is significantly higher than in non-travel sectors, due to the myriad combinations of origin and destination countries.
Turning an hypothesis into a successful promotional campaign is not an easy task and typically takes multiple iterations to optimize. There are multiple aspects which define a promotional campaign, but in general, those can be divided between settings or constraints and levers. The settings define the scope of the campaign, these are properties that cannot typically be modified and any campaign must comply with them, these include:
• Goal: The business objective of the campaign. it can be a certain amount of profit, increase in sales or user engagement, among others. • Budget: the finite resource that the company allows itself to invest in the campaign.
• Eligibility: The subset of OTP's traffic susceptible to the promotion. i.e in the case of cross-sell campaigns, only customers that already purchased a product can be targeted.
The levers are those aspects that can be tuned to optimize the promotional campaign, inside the framework defined by the constrains or settings. These include aspects of the campaign setup, and audience targeting. The former drives coarse changes in volumes and economics whereas the latter represents finer trade-offs between the two
The main levers of a promotional campaign set up are the following:
• Placement: Promotions can be offered at various stages, such as the upper funnel of the webpage, the lower funnel (post-purchase), or even through different contexts like marketing channels. The placement of the campaign predominantly affects campaign audiences and the quantity and quality of the customer information. Generally speaking the audience is reduced along the funnel and the amount of the customer information is increased. Therefore the campaign placement will determine target audience and the quantity and quality of the customer data available • Benefit type: A fixed discount is easier for the user to perceive and for predicting costs due its linear nature. However, in the travel industry, transaction value may vary significantly between different countries and destinations, thus making a fixed discount very appealing for cheaper destinations, where the expected revenue is small, making it economically inefficient. On the other hand, a percentage discount, or a benefit with monetary equivalent which is proportional to the transaction value, e.g., free breakfast, is more efficient across destinations but actual costs can be harder to predict. A fixed discount is therefore more suitable where the expected transactions value and/or expected revenue are relatively similar across items. • Benefit size: This is the actual amount or percentage of discount. The law of demand suggests that for most goods, there's an inverse relationship between the price and demanded quantity, or, in the context of promotions and benefits, the higher the benefit size, the higher the demand. However, increasing benefit size is very costly, and is also suffering from diminishing returns. Once the target audience and discount type are set, setting the sweet spot for discount value or benefit size would be such that the incrementality is maximized with economics (iROI, cost of acquisition, etc.) slightly under-performing, leaving room for further optimization [3].
In the context of promotional campaigns, the customer audience can be divided in four segments according to their response to a given discount [6].
• Complier: customers who respond positively only when targeted • Always-taker: Customers who respond positively, regardless of targeting • Never-taker: customers who won't respond regardless of being targeted.
• Defier: customers who are less likely to respond positively if they are targeted Figure 2 illustrates graphically the different segments described above The ideal targeting, in the context of promotion and benefits, is giving the discount or benefit only to the complier group. This way the promotion budget would be allocated in the most effective way. To that end, Uplift models are commonly employed to predict the causal effect of a treatment at the individual level based on data collected from a Randomized Control Trial (RCT) [7]. In uplift modeling, the incremental response of a customer with covariates 𝑋 is given by the Conditional Average Treatment Effect (𝐶𝐴𝑇 𝐸(𝑥)), defined by the difference in outcome had the customer been treated and not. high values of 𝐶𝐴𝑇 𝐸(𝑥) correspond to high incremental response. Therefore the goal of uplift modelling is to model the 𝐶𝐴𝑇 𝐸(𝑥) per customer given the available information (covariates 𝑥). There are several techniques to estimate the 𝐶𝐴𝑇 𝐸(𝑥), to name just a few:
• Meta-Learners is a family of standard ML models that are combined or modified to predict 𝐶𝐴𝑇 𝐸(𝑥). These include, the single model (S-Learner) [8] and two model estimator (T-Learner) [9]among others. • Uplift Trees and various deep learning based approaches, both uses modified loss functions of ML algorithms to predict 𝐶𝐴𝑇 𝐸(𝑥) [10]. • Retrospective Estimator is a technique that uses data from converted-only users to predict a proxy quantity to 𝐶𝐴𝑇 𝐸(𝑥) [11]. The main advantage of this technique is that it only requires data from customers that did materialize a purchase, which makes it very suitable for promotions placed at high end of the funnel. This technique has been proven to be highly beneficial in an online setting for multiple promotional use cases at Booking.com.
In an online setting, such as in E-Commerce platforms, a decision rule is required, the estimated 𝐶𝐴𝑇 𝐸(𝑥) is compared versus a threshold to decide whether to give a specific promotion to the customer or not. Using 𝐶𝐴𝑇 𝐸(𝑥) as a sorting mechanism, one could then evaluate the cumulative economics (iROI, Cost of acquisition, etc.) associated with treating all customers with 𝐶𝐴𝑇 𝐸(𝑥) >= 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑, also known as QINI Curve. It is customary to bucketize 𝐶𝐴𝑇 𝐸(𝑥) by population density, e.g., percentiles, deciles instead of equally spaced bucket, which allows to easily set the threshold that balances economics and incrementality.
A/B testing is the standard methodology to evaluate any new promotion campaign [12,13]. It is a well-established method where incoming traffic is randomly assigned to one out of two versions -'benchmark' (A) and 'experiment' (B) which are compared over a fixed period of time to determine which one performs better based on specific metrics. Once a given policy proofs to be superior in the A/B test it might be deployed as the new benchmark. While effective and well-established, this method has several limitations:
• Time-Consuming: Each test cycle requires distinct setup, execution, and analysis phases, leading to slower innovation cycles. • Lack of Flexibility: Once the test is set, variations do not change until the test concludes, which can limit the ability to adapt to new insights during the testing period.
Alternatively, continuous experimentation is a modern approach that offers ongoing, iterative testing and optimization which is specially well-suited. for the OTP's, given the seasonality of the travel market. Continuous experimentation can be defined as a different paradigm in which both experimental features and consolidated policies are continuously running. All the running policies conform a portfolio that is continuously monitored and modified if needed. This method offers several key advantages:
• Flexibility and Adaptability: Continuous experimentation allows real-time adjustments to test parameters and variations. Leveraging adaptive algorithms and real-time data analysis facilitates dynamic refinement based on user interactions, ensuring the testing process evolves in response to emerging insights. • Continuous baseline. By continuously allocating traffic to a baseline, campaign managers can monitor and detect seasonality effects or any disruption in the market. • Scalability and Comprehensive Insight: The methodology supports concurrent testing of multiple variations, providing a holistic view of user preferences and interactions. This capability enhances the depth of analysis and facilitates informed decision-making regarding feature optimization and deployment strategies.
Typically, in a continuous experimentation set up, we are working with three distinct sets of policies: Baseline, Benchmark and Experimental Policies. Each one is designed to test the effectiveness of discount strategies in a controlled and systematic manner. Below is a detailed explanation of each approach:
1. Baseline -a control group of traffic that does not receive any discounts or treatments. This group remain untreated to provide a baseline for comparison. 2. Benchmark -Well established promotion campaigns. 3. Experimental Policy-Any experimental policy to be tested.
Portfolio Average (Arbitrary metric)
While each of these policies is important on its own, one of the significant advantages of using this framework is the ability to manage all of them together as a portfolio of discounts strategies. Similar to a stock portfolio, this framework enables us to create a mixture of policies with different characteristics. Continuous monitoring of the portfolio allows promotional campaign managers to steer the portfolio. Steering refers to the change of the amount of traffic allocated to each of the policies of the portfolio in order to bring the portfolio average performance to the desired value. Figure 3 illustrates a steering scenario in which traffic is reallocated from one policy to another. This figure highlights two pivotal steering points where the portfolio performance metric is adjusted in response to shifts in economic or business priorities.
This work highlights the critical aspects of promotional campaigns in the online travel industry. We describe the unique characteristics of this domain, such as seasonality and interconnected verticals. We elaborate on the different features of a promotional campaign, emphasizing the distinction between constraints and levers, and list several options for leveraging the levers, such as targeting compliers. Additionally, we discuss the need for continuous experimentation and metric monitoring in contrast to traditional A/B testing. Finally, we emphasize the benefits of managing all promotions together as a single portfolio, which allows for balanced experimentation and alignment with business goals through strategic steering.