Toronto Raptors: Machine Learning as a Method for Improving the Roster Decision Process
In the competitive world of professional basketball, organizations such as the Toronto Raptors are turning to AI to improve the roster decision process in order to gain a competitive advantage in the race for a championship team.
A Need for a Competitive Edge in Basketball
Is there a role for machines when it comes to managing rosters in sports? In the 2002 Major League Baseball season, Billy Beane of the Oakland A’s, searching for a competitive advantage, laid the groundwork for this question by adopting a system of drafting and signing free-agents through data analytics, breaking conventional baseball wisdom along the way [1]. Moneyball, as it was termed, was controversial and ushered a new job title within professional sports, director of analytics [2]. 16 years later, the Golden State Warriors of the National Basketball Association (“NBA”) are dominating the sport and credit much of their success to Moneyball tactics, growing from a $300M valuation to $2.2B in eight years [3]. Team executives in the NBA are under pressure from fans and owners alike to replicate this level of success. Masai Ujiri, President of the Toronto Raptors, believes his team has found an edge by efficiently and innovatively managing their roster by taking Moneyball tactics to a new level with machine learning [4].
Artificial Intelligence as a Roster Management Tool
Roster decisions for the Toronto Raptors are managed by Ujiri and his personnel team. The roster creation process begins in the offseason with an amateur draft and free-agent signings and continues through the season with trades and related roster moves. Prior to 2016, Ujiri’s team curated and analyzed copious amounts of data, often in Excel, to form player target lists. Ujiri described this process as too manual and bogged down with paperwork [5].
Seeking to eliminate inefficiencies, the Raptors partnered with IBM Watson in 2016 to become the first NBA team to use the supercomputer to analyze players, removing cumbersome tasks and granting the Raptors more decision making time. In partnership with IBM Interactive Global Business Services, the Raptors built a state-of-the-art “War Room”, a center outfitted with multiple screens showing data captured from advanced cognitive technology from Watson [5].
In the short-term, the Raptors are using the War Room to integrate many of the same Moneyball principles as before but in real time and more comprehensively and innovatively. Traditional Moneyball tactics are prone to missing trends or players. Watson, however, can analyze large sets of structured and unstructured data related to the Raptors play, compiling both team and individual player efficiency metrics to pick up on trends over a larger set of data. This enables Watson to determine team deficiencies and ineffective players. Watson then matches these deficiencies and roster gaps with data on college players and free agents to identify players with the highest chances of improving the team in specified areas. Watson goes beyond simply reviewing player stats and analyzes videos more quickly and efficiently than anyone on Ujiri’s team can to identify strengths and style of play, then combines these insights with scouting reports, medical histories and personality evaluations derived from interviews and social media for cultural fit to make player recommendations [6]. In time-pressured situations such as the draft, Watson delivers all of this in a matter of seconds compared to the former process of calling statisticians to get slimmer versions of recommendations [5].
In the medium-term, these recommendations are helping to drive roster management. Watson’s evaluation tool is fluid, a player who may have been graded as positive one day could be viewed negatively the next as Watson incorporates new scouting reports or an ill-advised social media post [5]. The Raptors first used Watson in the 2016 NBA draft and, although tight-lipped on results thus far for competitive reasons, have been working with the IBM team to continue making tweaks to improve the tool.
Additional Uses of Artificial Intelligence
Beyond the use of AI in roster management, the Raptors can further their edge in the short-term by incorporating Watson as a highly efficient assistant coach. The next stage of machine learning in sports is to formulate game strategies based on video of opponent tendencies and exploitable weaknesses. In the NBA, teams play every other night and often the game plan is put together by scouts [7]. Having Watson as a tool to comprehensively scout the other team in short order would provide meaningful advantages to the Raptors every night.
In the medium-term, an area to explore is player development. Using the same method behind player evaluations for roster construction, the Raptors could use Watson to devise individualized plans for player development. Watson could be used to measure potential and recommendations on player development goals and steps.
Open Questions
Watson provides recommendations based on criteria fed by the Raptors, leaving recommendations susceptible to some human biases. Also, one could view the Raptors as IBM’s Guinea and sell solutions to other teams in the future. Given these factors, is there enough reason to believe that machine learning truly gives the Raptors a competitive edge?
Words: 799
Sources:
[1] Mike Piellucci, “An Experiment That Changed Baseball: The Moneyball Draft 15 Years Later”. Vice.com, June 12, 2007. [https://sports.vice.com/en_us/article/pay3n9/an-experiment-that-changed-baseball-the-moneyball-draft-15-years-later]. Accessed November 2018.
[2] Steve Levine, “AI is Getting a Tryout in Pro Sports”. Axios.com, September 17, 2017. [https://www.axios.com/ai-is-getting-a-tryout-in-pro-sports-1513305710-ff256a96-72e5-41aa-b7f2-2b95bbd9af9d.html]. Accessed November 2018.
[3] Andrea Powell, “How Moneyball Tactics Built a Basketball Juggernaut”. Wired.com, November 2, 2017. [https://www.wired.com/story/how-moneyball-tactics-built-a-basketball-juggernaut/]. Accessed November 2018.
[4] IBM Case Studies Website. [https://www.ibm.com/watson/stories/ca-en/basketball-with-watson.html]. Accessed November 2018.
[5] Paolo Del Nibletto, “Toronto Raptors Unveil a Digital War Room”. ITWorldCanada.com, February 11, 2016. [https://www.itworldcanada.com/article/toronto-raptors-unveil-a-digital-war-room/380686]. Accessed November 2018.
[6] Ron Miller, “IBM Watson Teams with Toronto Raptors on Data-Driven Talent Analysis”. Techcrunch.com, 2015. [https://techcrunch.com/2016/02/10/ibm-watson-teams-with-toronto-raptors-on-data-driven-talent-analysis/]. Accessed November 2018.
[7] Doug Brotherton, “Five Scouting Tips from the NBA to High School”. Fastmodelsports.com, November 15, 2017. [http://team.fastmodelsports.com/2017/11/15/scouting-nba-high-school/]. Accessed November 2018.
It’s interesting to hear the ways in which the Raptors have quickly leveraged IBM Watson’s capabilities to inform their talent acquisition strategy. It seems like IBM’s technology is effective in providing the Raptors scouting team with a more holistic view of 1) their team’s performance and potential gaps, and 2) individual player potential – based on a much wider array of data. One concern I have is that this feels like a static view. If the Raptors are assessing their performance gaps at a point in time, can machine learning effectively help to predict how those gaps may change over time? In other words, can IBM Watson predict player-team compatibility based on potential future scenarios?
Super insightful article! It’s fascinating to see machine learning applied in sports. I do, however, agree with your comment that the results of the analysis are inherently subject to human biases in the decision making process. There is a tremendous amount of subjextivity in player evaluation, however it can clearly be used as a tool to help influence one’s decisions.Great job!
#wethenorth
Bo G – very nice article. The Toronto Raptors are a very interesting test case in my mind given their recent trade for Kawhi Leonard, an all-NBA player who has lingering questions about his health and attitude. The article mentions that Watson is able to screen interviews and social media to get a read on a player’s cultural fit, but I wonder (i) how well it can actually do this and (ii) how it could account for Kawhi’s questionable health. Separately, I wonder how well Watson works in the context of the NBA in which adding a player like Kawhi materially affects the rest of the team (in contrast to baseball where each player operates in a mostly independent manner). If anything, these questions point to why there is a continued need for human oversight – Watson can be seen as more of an unbiased viewpoint rather than the source of truth for NBA GMs.
Mark, great example of an area where the data input to Watson might not be able to pick up on the intangibles of players. I would go further and think that Watson will miss a lot of players who are not the “stars” in college with the highest stats, due to the lack of data input to predict potential. I foresee Watson as being a tool to add one more data point in a decision, versus the machine that will make the decisions. It will be interesting to see how the NBA and other professional sports continue to incorporate data and machine learning into their strategy.
As an avid NBA fan, I really enjoyed reading this! I didn’t know that the Raptors used IBM Watson to scout and draft players so this was fascinating to read and I thought your comment about using Watson for player development was really good. In the 2016 draft, they took Poeltl at 9 and Siakam at 26 and in 2017 they took Anunoby at 23. Overall, based on the other players that were available at these draft positions and the growth of those other players over the past couple years, I’d give the Raptors a B grade. There’s certainly a lot of room to improve their analytics model.
My concern about using IBM Watson is that it relies on historical data. Over the past several years, the style of play in basketball has gone through a fundamental shift, now much more free-form with pace and space, more 3’s, and less defense. I worry that IBM Watson wouldn’t have been be able to anticipate such a radical shift in style of play and that it would have recommended players based on incremental change that would have been expected. This is like in the GAP marketing case where machine learning in the fashion industry wasn’t able to pick up really drastic revolutionary shifts in fashion trends. To be ahead of the curve and create something brand new, human guidance was needed. Would IBM Watson have predicted the rise of the Warriors or of the Rockets’ mind-boggling statistical three-point season last year?
Finally, this application raises a few additional questions for me: 1) how do they account for players who will make an immediately impact vs. players who will need further development but have higher long-term potential; 2) how do they build into their analytics model the skills/ability of the coaches to actually develop young players; 3) will this machine learning arms race fundamentally change the way basketball is played and enjoyed by fans?
Thanks for writing about this. If / when other teams start partnering with IBM Watson, I believe the aspects mentioned of “scouting reports, medical histories and personality evaluations derived from interviews and social media for cultural fit” will become the primary differentiators in teams’ judgments of certain players; machine learning is taking out the human judgment for the quantitative measures of a player’s performance.
The use of machine learning in coaching is fascinating. Could there be one day no coaches, and only an algorithm that recommends precise lineups and plays? I don’t think this will manifest – there is still a human element of a leader in sports that is valuable in motivating a team.
This post reminded me of the Jacksonville Jaguars, who has a robust analytics department (https://www.jacksonville.com/news/20180831/jaguars-analytics-teams-offers-insights-into-questions-on-field-and-in-stands). I’d be curious to see how the Toronto Raptors uses machine learning not only on the court, but off the court for the fan experience.