Improving Uber User Experience Through Causal Inference Analysis

How one of the world’s largest ridesharing and food delivery companies is leveraging causal inference methods to acquire richer insights into operations analysis, product development, etc.

A large portion of Leading with People Analytics (LPA) has been focused on regression analysis, a common approach in statistics in which we are concerned with quantifying how changes in variable x are associated with changes in variable y. Another increasingly popular family of methods within the field of statistics is causal inference analysis, in which we are instead concerned with determining whether changes in variable x cause changes in variable y. Many industry engineers and researchers are showing interest in this growing sub-field of statistics, as it offers a more principled approach to understanding the causes behind the results that we see from observations or experiments. Uber, one of the world’s largest ridesharing and food delivery companies, has already formed a causal inference community whose task is to apply causal inference methods that bring richer insights into operations analysis, product development, and other areas critical to improving the user experience.

In the article, “Using Causal Inference to Improve the Uber User Experience,” Harinen and Li layout what is causal inference, why it is important, and how Uber engineers are using various causal inference methods to solve critical data science questions. For example, those at Uber Labs are interested in how experiencing an event like a delay in food delivery can influence customers’ future engagement with the Uber Eats platform. Answering this question requires working with observational data, as running an experiment under the given circumstances is infeasible since it would negatively impact the customer experience. As we’ve discussed during class, simply calculating the difference in future customer engagement between users who experienced a delay versus those who did not would likely not result in a meaningful answer due to the presence of potential sources of confounding such as the number of customer food orders. Therefore, Harinen and Li highlight how Uber engineers are using causal inference methods such as propensity score matching to account for sources of confounding and achieve more causal estimates of treatment effect. This is one of many illustrations of the use of causal inference methods at Uber Labs provided throughout the article. Another particularly interesting example is the use of regression discontinuity to investigate how different levels of dynamic pricing influence customers’ decisions to request a ride on the Uber platform.

Overall, as a data science student who is particularly interested in the various applications of causal inference methods, it is exciting to see that high-profile companies such as Uber are using these methods to better inform decision-making. Although I am a major proponent of the use of causal inference analysis to investigate interesting questions in data science, I am also aware of its limitations. The validity of the estimates achieved using causal inference methods often rests on certain untestable assumptions. Ensuring that these assumptions are reasonable requires a high level of domain expertise. Therefore, I’d imagine that the engineers carrying out causal inference analysis at Uber Labs are in close collaboration with the domain experts who have substantive knowledge of the business problems at hand. I am also curious as to whether Uber is using causal inference analysis to investigate its own employee data. For instance, one interesting application that comes to mind involves the use of regression discontinuity to study how company-level policy changes affect employee satisfaction. There are surely various other applications of causal inference methods, and it will be interesting to learn more as Uber and other major technology companies continue to uncover them.

SOURCE: Harinen, Totte, and Bonnie Li. “Using Causal Inference to Improve the Uber User Experience.” Uber Engineering, 19 June 2019. https://eng.uber.com/causal-inference-at-uber/.

Previous:

Trust Issues: Obstacles to Using People Analytics for DEI Efforts

Next:

Take Step 1 back to square 1

Student comments on Improving Uber User Experience Through Causal Inference Analysis

  1. Hi Joeseph,

    Great article, thanks for sharing. I think you touch on a lot of exciting areas in prediction modeling and I think the ones around causal inference are particularly important. I feel like we are reaching a stage where there’s a lot of regression analysis being done in determining the association between two or more variables, but due to limitations in hypothesis testing often we cannot infer whether there is a causal link between the two or more variables we are testing. This is where we get into trouble by making false causation assumptions (like chicken consumption causes changes in oil consumption). I think it’ll be interesting to see how much weight Uber and companies like them place on this type of analysis. Theoretically, if you can prove that something has a causal link, it would be only fair to therefore invest a significant amount of resources into the causal agent. However, as you point out, there are some assumptions that are being made and if Uber starts gambling on these analyses where the limitations of the assumptions are not being fully appreciated, there could be some costly mistakes!

  2. I appreciated your very easy to understand explanation regarding causal inference analysis as compared to the regression analyses we’ve been discussing in class! I find this application very exciting as a user experience designer. This type of data could be so influential in how information and interactions are laid out within a product. This application of people analytics is also a bit different than much of what we’ve discussed in class because it is performing the analysis externally on customers, rather than internally in an HR capacity.

    Additionally, I wonder if this is the same type of analysis that Netflix uses for their cover images within the app. Netflix personalizes the image of each show/movie based on your specific profile and what it feels you’ll be most drawn to. I know Netflix is very big into experimentation and data analysis, so I’m curious if this specific type of analysis is occurring to identify if, at a statistically significant level, cover image selections are causing a change in customer viewing selections.

Leave a comment