Existing statistical ultimate (Frisbee) analyses rely on data aggregates to produce numeric statistics, such as completion percentage and scoring rate, that assess strengths and weaknesses of individuals and teams. We leverage sequential, location-based data to develop completion and scoring maps. These are visual tools that describe the aggregate statistics as a function of location. From these maps we observe that player and team statistics vary in meaningful ways, and we show how these maps can inform throw selection and guide both o ensive and defensive game planning. We validate our model on real data from highlevel ultimate, show that we can characterize both individual and team playing, and show that we can use map comparisons to highlight team strengths and weaknesses.
The growth of ultimate (Frisbee) in numbers and maturity is leading to rapid changes in the eld of ultimate statistics. Within the past several years, tracking applications on tablets have begun providing teams with information about the performance on an individual and group level, e.g., through completion percentages. As these applications are maturing, the application developers need feedback to identify better ways to collect data so that the analysts can query the application to help them guide team strategy and individual development. Likewise the analysts must interact with team leadership to identify the answerable questions that will result in bene cial adjustments to team identity and game approach.
Existing ultimate statistics are akin to the baseball statistics used prior to
the spread of sabermetrics[
To address some of these shortcomings, rst we introduce completion and
scoring maps for the visualization of location-based probabilities to help capture
high-level strategy. Similar shifts towards visual statistics are taking place in
other sports, e.g., in basketball[
Eisenhood reveal team strengths and weaknesses and provide a comparison between teams. Second, we show how these maps can be used to shape individual strategy by recommending optimal throw choices. Finally, we discuss how the maps could be used to guide defensive strategy.
We review the basics of ultimate and ultimate statistics in Section 2. In Section 3 we introduce our visual maps for scoring and completion. In Section 4 we move from conceptual maps to maps based on empirical data. We discuss use cases for maps in Section 5 and o er a broader discussion for continued improvement in statistical analysis of ultimate in Section 6. 2
To begin, we review the basic rules of ultimate. Ultimate is a two-team, sevenon-seven game played with a disc on a rectangular pitch with endzones, and the goal of the game is to have possession of the disc in the opponent's endzone, i.e., a score. The player with the disc must always be touching a particular point on the ground (the pivot) and must release the disc within 10 seconds. If the disc touches the ground while not in possession or the rst person to touch the disc after its release is the thrower, the play results in a turnover and a member of the other team picks up the disc with the intent of scoring in the opposite endzone. Each score is worth one point, and the game typically ends when the rst team reaches 15.
Collecting statistics can help teams understand the skills and weaknesses of their players and strategies. Table 1 shows some statistics kept to help assess player strengths and weaknesses. While these aggregate statistics can be useful, visual statistics and analyses o er a complementary characterization of individual and team ability. In addition to tabulating player and team statistics over games, we collect locations of throws and catches, giving us location- and sequence-speci c data. 3
Let us consider a game of ultimate. A game is a sequence of points, which are sequences of possessions, which themselves are sequences of plays (throws). Each play, a thrower, a receiver (if any), and their respective locations are recorded. 60 50
It is recorded if a turnover occurs and speci cally whether the turnover was due to a block, an interception, or a throwaway. If a completion occurs in the opponent's endzone, a score is recorded.
We introduce a model over throws 2 T , where throws are speci ed by players x and locations z. Each player xi, for i = 1; 2; : : : n, completes a throw with probability p = p(x0; z0; x1; z1), where includes the tuple ((x0; z0); (x1; z1)), denoting that player x0 throws to x1 from location z0 to z1 on the pitch.
Given throws, we can construct a completion map. A completion map shows the probability of completion of a throw from player xi to receiver xj , based on the receiver location zj . A map is de ned for every starting location zi of every player xi. Figure 1 (left) provides a completion map for a player trapped on the sideline (blue dot). As is shown, long throws and cross- eld throws are the most di cult throws in ultimate (on average).
Chaining together throws, we de ne a path that corresponds to a sequence of throws 0; 1; : : : ; k for some k, where the superscripts denote the throw sequence index. Note the set of paths is countable but unbounded. We also make the Markov assumption that the probability of completing a pass p(xi; zi; xj ; zj ) is independent of all passes prior given xi and zi. We de ne a possession 0 as a path that ends in either a score (1) or a turnover (0). The probability of scoring starting with (x0; z0) is then: p(Scorejx0; z0) =
X 1[ 0]p( 0) =
X 1[ 0] Y p(xi; zi; xi+1; zi+1) (1) 0 0 where 1[ 0] equals 1 if the possession results in a score and 0 otherwise. Unfortunately the probability is di cult to compute, but we can approximate it by introducing the probability p(Scorejz0) = n1 Pxi p(Scorejxi; z0) that the team scores from a location z0 on the eld, which is the marginal probability of scoring over players. We will use this approximation in Section 5.
For now, we can use p(Scorejz0) to de ne our scoring map. A scoring map provides the probability that a team will score from a location z0 for every location z0 on the eld. As shown in Figure 1 (right), the probability of scoring is high when the disc is close to the opponent's endzone, and low when the disc is far away. From Figure 1 (right) we see it is also advantageous to have the disc in the middle of the eld. Ultimate experience suggests that such an increase in scoring probability exists because more in-bounds playing eld is accessible with short throws.
To foreshadow, we can use the completion and scoring maps in conjunction to better understand ultimate. We will use it recommend where to throw the disc, how to game-plan for high wind situations, and how to make defensive adjustments. First however, we use data and nearest neighbor methods to show that our model maps re ect existing ultimate beliefs. 4
While using simpli ed models to construct completion and scoring maps
(mixtures of Gaussians[
From the throws table, we produce empirical completion and scoring maps.
We do this using k-nearest neighbors[
Completion and scoring maps can be used directly as aids for players, but they also have other applications. First we show how maps can determine throw choice. Next, we apply throw choice to show how maps can be used to change o ensive strategy given external factors. Finally, we show how defensive ability to manipulate the completion map could guide defensive strategy. Throw choice The best throw a player can make is the one that maximizes the team's probability of scoring the point. Note that the probability of scoring the point is distinct from the probability of scoring the possession. In this section we relate the two using completion and scoring maps to look at the expected point-scoring outcomes by throw choice.
Recall from Equation 1 that we get the probability of scoring from considering all possible paths given (x0; z0). Now we can consider the probability of scoring based on where the player chooses to throw. The probability of scoring with a throw to (x1; z1) is given by the probability of completing the rst throw times 60 50 This approximation is assuming that the particular receiver of the rst throw x1 does not a ect the scoring probability. Using this approximation, we can use the completion map to get p(x0; z0; x1; z1) and the scoring map to get p(Scorejz1). By decomposing the probability, we now have an approximation to the probability of scoring given our throw choice.
To nd the (approximately) optimal throw, however, we should consider the expected value given throw choice, not the probability of scoring the possession. The expected value can be determined by assigning a value of +1 to a score, and -1 to an opponent score. To simplify further, let us assume that each team only gets one chance to score. If neither team scores in their rst possession, the expected value is 0. Then we can determine the expected value using the completion map, the scoring map, and the opponent scoring map. Figure 3 shows the expected value map given the maps from Figure 1. Then, we can nd the maximum expected value, and instruct the player to throw to that location.
Note that the di erence in expected values may seem small{just a fraction of a score. However, they add up. In the Nexgen games there were an average of 300 throws per game. If a situation presents itself, for example, 10 times in a game, choosing an action that makes a di erence of 0.1 each time results in scoring an extra point on average.
O ensive strategy in the wind External factors can govern the completion and scoring maps. For example, windy conditions lower the probability of completion, and thus the probability of scoring on a possession. By understanding or approximating changes in the probabilities, a team can change its mindset. The map in Figure 3 (right) shows the new expected value map if you lower the probability of completion in the completion and scoring graphs. While the original expectation graph in Figure 3 (left) recommended throwing a short pass (called a reset), in high wind the graph recommends a long pass (called a huck). Defensive strategy We noted that o ensive game-planning, e.g. should a team throw long, high-risk throws, is a ected by knowledge of location-based scoring probabilities. Similarly, defensive strategies will a ect the opponent completion and scoring probabilities as well. Using scoring maps that take into account defensive positioning, we could identify the minimax outcomes that govern optimal o ensive strategy given defensive strategy. That is, the defense should employ the strategy that results in the smallest maximum expected value on the map, and the o ense should choose the throw that maximizes the expected value on the map. 6
The analysis presented highlights the capabilities of completion and scoring maps. Many other uses of the maps and location-based data would be interesting. For one, we can use these maps to characterize players. We can answer questions such as: how do player completion rates change across the eld, and does the player have weaknesses or strengths in particular regions? We can also use the expectation maps in conjunction with the throws table to assess how much \value" each player contributes to the team by summing up the change in expected values for plays in which the player was involved. Furthermore, we can perform visual comparative analyses between teams (or players). Subtracting the scoring maps from one another (or a baseline) can help identify regions of relative weakness; the empirical comparative scoring map (di erence in maps in Figure 2) is shown in Figure 4.
While location-based tracking adds a visual and predictive component that helps describe optimal ultimate play, it does not provide other pertinent information that teams and players must address when making decisions on the eld. For example, the location of players not involved the movement of the disc a ect the choices made. An alternative analysis could track not only the disc movement but all 14 player's movements. Also, while our analysis uses the sequence of throws (to determine possessions and scores), the analysis is atemporal. Many throws are relatively easy o of disc movement because the defenese is out of 0.3 0.2 0.1 0.0 −0.1 −0.2 −0.3 position, and an atemporal model does not capture these elements of the game. Another important factor, weather condition, goes unmodeled. Incorporating these into our models would help re ne our analysis and additional insight into ultimate strategy. Finally, the number of throws available for analysis will always be relatively small, particularly against uncommon strategies or in unusual conditions. Developing strategies and assessing individual ability in the face of limited data will be challenging and should be considered in ultimate analyses.
We would like to acknowledge Ultiworld and UltiApps for their continued support and development of ultimate statistics.