Not just winging it: predicting airfare at KAYAK
How KAYAK uses machine learning to predict drops in plane ticket prices.
It’s an urban legend with surprising ubiquity: the best time to buy plane tickets is Tuesday. While that may be a myth [1], consumers still go to extreme measures to find deals among airfare fluctuations [2]. Today, companies are using machine learning to predict drops in ticket prices and deliver value to travelers.
The online travel industry is comprised of three tiers: direct suppliers like Marriott or JetBlue, which sell and provide travel services; online travel agencies (OTAs) such as Orbitz or Expedia that aggregate supplier offers and facilitate reservations; and metasearch companies like KAYAK and Google that compare prices across supplier sites and OTAs. In 2017, an estimated $700 billion in travel reservations were made online [3], about half of which was on airfare [4].
In this competitive space, companies aiming to convert travel-searchers into travel-bookers leverage machine learning to provide differentiated online experiences. By discerning booking patterns in past searches, travel sites modify their result rankings to increase the likelihood of a reservation. However, online travel sites have long struggled to crack one of travelers’ most pressing questions: when is the best time to book a plane ticket?
Volatility in airfare is influenced by fluctuations in supply and demand, plus changes in competition, fuel prices, regulation, and geopolitical conditions. KAYAK, the travel metasearch subsidiary of Booking Holdings, is a pioneer among companies attempting to forecast changes in airfare. Since 2013, KAYAK has been developing machine learning models that power a recommendation engine it asserts is more agile and accurate than those offered by competitors like Hopper, Skyscanner, or now-defunct Fareceast [5].
Because KAYAK is a metasearch engine already hyper-focused on comprehensiveness, its access to data from a wide breadth of providers offers a critical advantage in training the models. The KAYAK price prediction model considers data from millions of flight searches; it analyzes search parameters to anticipate the probability of a route’s airfare rising or falling in the next seven days given patterns among similar historical flights. The fare predictor then suggests the traveler either purchase now or wait for a price drop [5]. The prediction applies to all flights for the searched route and dates; the model does not make predictions for individual flight numbers or fare classes. In presenting fare predictions to users, KAYAK must communicate both the recommendation (buy or wait) and the level of confidence in that result.
Onsite fare prediction messaging
Source: KAYAK.com, “Price Trends and Tip Explanation,” https://www.kayak.com/price-trend-explanation, accessed November 2018.
In deploying airfare predictions, KAYAK must contend with tradeoffs during product development: a price-predictor, when perceived as accurate and trustworthy, creates substantial value for customers. But advising customers to postpone purchasing introduces the chance that they ultimately book on a different website. Furthermore, the product team must design for an online experience that works whether or not the model returns a confident prediction, plus decide at what confidence threshold to display a recommendation, understanding that frequency of displaying predictions impacts the perceived trustworthiness of the tool. This presents a particular challenge: accuracy of displayed predictions is inversely correlated with frequency of displayed predictions.
KAYAK must also consider the level of precision and granularity it employs in its recommendations. Today, predictions are route-specific; would flight-specific predictions offer value, or be an exercise in diminishing returns? And finally, the team must develop methods of evaluating these design choices, while considering the impact of Type 1 (model recommends purchasing now but the price drops later) and Type 2 (model recommends waiting but the price rises) errors on customer satisfaction. Today, KAYAK continually assesses the efficacy of its model by comparing its predictions to fare fluctuations actually observed. It tests changes to algorithm design and user interface through concurrent A/B testing [6].
When to book plane tickets
Source: KAYAK.com, “When to Book,” https://www.kayak.com/travelhacker/2017/when-to-book, accessed November 2018.
KAYAK embeds machine learning across its product suite. Recently, KAYAK parlayed its effort in natural language processing for a chat-based booking tool [7] into an Amazon Alexa skill [8]. It also announced an augmented reality feature in its app that detects whether luggage meets carry-on size requirements [9]. To integrate machine learning in its development process, KAYAK situates data scientists in product development teams, encouraging bottom-up ideas.
As KAYAK looks to the future, the team must consider risks to the effectiveness of its models. The company’s 2017 10-K filing points out that maintaining relationships with suppliers and OTAs is critical to their access to data [11]. Furthermore, they must invest in computational infrastructure that will enable them to compete with fast and precise predictions currently being tested by competitors. Finally, KAYAK must develop means to detect user behavior cross-device as more bookings move to mobile platforms.
As it looks to enhance its machine learning product features, KAYAK must focus both on the efficacy of its models and on the user experience of delivering predictions. By engendering trust in its airfare forecasts, KAYAK lays the foundation for delivering predictions across its other verticals, such as hotels and rental cars.
[1] Jenkins, Aric, “Cheap Flights: Tuesday Not the Best Day to Buy Plane Tickets | Money,” Time, July 05, 2015, http://time.com/money/4845914/tuesday-not-cheapest-day-to-buy-plane-tickets/, accessed November 2018.
[2] Gabrielse, Zeke, “The Programmer’s Guide to Booking a Plane,” Hacker Noon, December 20, 2016, https://hackernoon.com/the-programmers-guide-to-booking-a-plane-11e37d610045, accessed November 2018.
[3] eMarketer, and Tnooz, Digital travel sales worldwide from 2014 to 2020 (in billion U.S. dollars), https://www-statista-com.ezp-prod1.hul.harvard.edu/statistics/499694/forecast-of-online-travel-sales-worldwide/, accessed November, 2018.
[4] Nöldeke, Geeske, ETravel Report 2018 – Online Travel Booking, Report, Hamburg: Statista, 2018, https://www-statista-com.ezp-prod1.hul.harvard.edu/outlook/262/100/online-travel-booking/worldwide, accessed November 2018.
[5] Schaal, Dennis, “Skift Tech Forum Preview: Kayak’s Chief Scientist Confident on Fare Predictions,” Skift, June 05, 2018, https://skift.com/2018/06/05/skift-tech-forum-preview-kayaks-chief-scientist-confident-on-fare-predictions/, accessed November 2018.
[6] Thomke, Stefan, and Daniela Beyersdorfer, “Booking.com,” HBS No. 619-015 (Boston: Harvard Business School Publishing, 2018).
[7] Schaal, Dennis, “Kayak to Launch New Text-Based Travel Service,” Skift, December 24, 2015, https://skift.com/2015/12/23/kayak-to-launch-new-text-based-travel-service/, accessed November 2018.
[8] Sorrells, Mitra, “Kayak Adds Flight Notifications to Alexa Skill,” PhocusWire, April 19, 2018, https://www.phocuswire.com/Kayak-Alexa-flight-notifications, accessed November 2018.
[9] Perez, Sarah, “Kayak’s New AR Feature Will Tell You If Your Carry-on Bag Fits the Overhead Bin,” TechCrunch, September 18, 2018. https://techcrunch.com/2018/09/18/kayaks-new-ar-feature-will-tell-you-if-your-carry-on-bag-fits-the-overhead-bin/, accessed November 2018.
[10] Kniberg, Henrik, “Spotify Engineering Culture (part 1).” Spotify Labs, September 20, 2014, https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/, accessed November 2018.
[11] Booking Holdings, Form 10-K 2017, p. 12, http://ir.bookingholdings.com/financial-information/annual-reports, accessed November 2018.
Very interesting post! This seems to be an excellent application of machine learning that is already delivering value to its customers. KAYAK’s point on the importance of their relationships with suppliers and OTAs is critical for their machine learning programs to continue to function successfully into the future. As we have learned, machine learning is only effective when there is access to good data, so KAYAK should in the short and long term work to maintain these relationships with these suppliers to ensure that all parties continue to get value from their services. In response to your questions, I do think KAYAK needs to continue to devote a significant amount of its resources to the airline predictions so that they maintain a competitive edge over others. It would also seem very natural for them to be able to impose a similar model on the hotels and rental car services, provided that they have the same access to good data.
This is a fascinating article. The price and timing to buy plane tickets is always a topic of discussion when planning a trip. By creating this prediction analysis, it gives the customer a confidence to either wait or purchase the ticket right away. However, as this prediction contradicts the reality, the level of confidence in the website’s analysis declines rapidly. Hence, it’s really important to try to minimize the errors in the predictions and only share them when the level of confidence of the machine is really high.
Great article! I didn’t know about this functionality of KAYAK but I immediately checked it out and in fact I just booked my winter tickets because of it!
I think this algorithm gives KAYAK a great competitive advantage and in my opinion, the company should heavily focus on its further improvement.
One disadvantage that I noticed is that for complex, multi-city routes, it takes quite some time to come up with a recommendation (<10 min). So investing in computational power, especially as more and more data become available (and hopefully as more and more customers use the website) computational power requirements will increase.
Also, if the company is confident that it can find a way to expand this feature in hotels and car rentals, I would be a highly enthusiastic supporter of the expansion!