=Paper=
{{Paper
|id=Vol-1969/paper-04
|storemode=property
|title=Maps for Reasoning in Ultimate
|pdfUrl=https://ceur-ws.org/Vol-1969/paper-04.pdf
|volume=Vol-1969
|authors=Jeremy Weiss,Sean Childers
|dblpUrl=https://dblp.org/rec/conf/pkdd/WeissC13
}}
==Maps for Reasoning in Ultimate==
https://ceur-ws.org/Vol-1969/paper-04.pdf
Maps for Reasoning in Ultimate
Jeremy C. Weiss1 and Sean Childers2
1
University of Wisconsin-Madison, Madison, WI, USA
2
New York University, New York City, NY, USA
Abstract. Existing statistical ultimate (Frisbee) analyses rely on data
aggregates to produce numeric statistics, such as completion percentage
and scoring rate, that assess strengths and weaknesses of individuals and
teams. We leverage sequential, location-based data to develop completion
and scoring maps. These are visual tools that describe the aggregate
statistics as a function of location. From these maps we observe that
player and team statistics vary in meaningful ways, and we show how
these maps can inform throw selection and guide both offensive and
defensive game planning. We validate our model on real data from high-
level ultimate, show that we can characterize both individual and team
playing, and show that we can use map comparisons to highlight team
strengths and weaknesses.
1 Introduction
The growth of ultimate (Frisbee) in numbers and maturity is leading to rapid
changes in the field of ultimate statistics. Within the past several years, track-
ing applications on tablets have begun providing teams with information about
the performance on an individual and group level, e.g., through completion per-
centages. As these applications are maturing, the application developers need
feedback to identify better ways to collect data so that the analysts can query
the application to help them guide team strategy and individual development.
Likewise the analysts must interact with team leadership to identify the answer-
able questions that will result in beneficial adjustments to team identity and
game approach.
Existing ultimate statistics are akin to the baseball statistics used prior to
the spread of sabermetrics[1]. Table 1 shows some of the basic statistics kept on
individuals. A similar table is kept for team statistics. These data are relatively
easy to capture using a tracking application, as sideline viewers can input the
pertinent information as games progress. However, they lose much in terms of
capturing the progression of the game and undervalue certain player qualities,
for example, differentiating between shutdown defense and guarding idle players
(both result in low defensive statistics).
To address some of these shortcomings, first we introduce completion and
scoring maps for the visualization of location-based probabilities to help capture
high-level strategy. Similar shifts towards visual statistics are taking place in
other sports, e.g., in basketball[2]. Completion and scoring maps can be used to
Player Points Throws Completions % Goals Assists Blocks Turnovers
Childers 65 88 100 0.88 20 4 10 8
Weiss 40 49 50 0.98 2 10 4 1
Eisenhood 55 20 30 0.67 10 1 20 15
Table 1: Table of the ultimate statistics collected on individuals. Aggregates
statistics for teams use similar fields.
reveal team strengths and weaknesses and provide a comparison between teams.
Second, we show how these maps can be used to shape individual strategy by
recommending optimal throw choices. Finally, we discuss how the maps could
be used to guide defensive strategy.
We review the basics of ultimate and ultimate statistics in Section 2. In
Section 3 we introduce our visual maps for scoring and completion. In Section
4 we move from conceptual maps to maps based on empirical data. We discuss
use cases for maps in Section 5 and offer a broader discussion for continued
improvement in statistical analysis of ultimate in Section 6.
2 Background
To begin, we review the basic rules of ultimate. Ultimate is a two-team, seven-
on-seven game played with a disc on a rectangular pitch with endzones, and the
goal of the game is to have possession of the disc in the opponent’s endzone, i.e.,
a score. The player with the disc must always be touching a particular point
on the ground (the pivot) and must release the disc within 10 seconds. If the
disc touches the ground while not in possession or the first person to touch the
disc after its release is the thrower, the play results in a turnover and a member
of the other team picks up the disc with the intent of scoring in the opposite
endzone. Each score is worth one point, and the game typically ends when the
first team reaches 15.
Collecting statistics can help teams understand the skills and weaknesses
of their players and strategies. Table 1 shows some statistics kept to help as-
sess player strengths and weaknesses. While these aggregate statistics can be
useful, visual statistics and analyses offer a complementary characterization of
individual and team ability. In addition to tabulating player and team statistics
over games, we collect locations of throws and catches, giving us location- and
sequence-specific data.
3 Completion and scoring maps
Let us consider a game of ultimate. A game is a sequence of points, which are
sequences of possessions, which themselves are sequences of plays (throws). Each
play, a thrower, a receiver (if any), and their respective locations are recorded.
1.0 1.0
0.4
0.9
0.8
60 60
0.7
0.8 0.8
0.5 0.6
50 50
0.4
0.5
0.6 0.6
Sideline
40 40
30 0.6 30
0.4 0.4
20 0.7 20
0.4
0.8 0.2 0.2
●
10 10
0.9
0.0 0.0
10 20 30 10 20 30
P(completion (handler)) P(score)
Fig. 1: Completion map (left) for a handler (a thrower) and scoring map (right).
The blue circle denotes the thrower location.
It is recorded if a turnover occurs and specifically whether the turnover was
due to a block, an interception, or a throwaway. If a completion occurs in the
opponent’s endzone, a score is recorded.
We introduce a model over throws τ ∈ T , where throws are specified by play-
ers x and locations z. Each player xi , for i = 1, 2, . . . n, completes a throw τ with
probability pτ = p(x0 , z 0 , x1 , z 1 ), where τ includes the tuple ((x0 , z 0 ), (x1 , z 1 )),
denoting that player x0 throws to x1 from location z 0 to z 1 on the pitch.
Given throws, we can construct a completion map. A completion map
shows the probability of completion of a throw from player xi to receiver xj ,
based on the receiver location zj . A map is defined for every starting location zi
of every player xi . Figure 1 (left) provides a completion map for a player trapped
on the sideline (blue dot). As is shown, long throws and cross-field throws are
the most difficult throws in ultimate (on average).
Chaining together throws, we define a path ρ that corresponds to a sequence
of throws τ 0 , τ 1 , . . . , τ k for some k, where the superscripts denote the throw
sequence index. Note the set of paths is countable but unbounded. We also make
the Markov assumption that the probability of completing a pass p(xi , z i , xj , z j )
is independent of all passes prior given xi and z i . We define a possession ρ0 as a
path that ends in either a score (1) or a turnover (0). The probability of scoring
starting with (x0 , z 0 ) is then:
X X Y
p(Score|x0 , z 0 ) = 1[ρ0 ]p(ρ0 ) = 1[ρ0 ] p(xi , z i , xi+1 , z i+1 ) (1)
ρ0 ρ0 τ i ∈ρ0
where 1[ρ0 ] equals 1 if the possession results in a score and 0 otherwise. Un-
fortunately the probability is difficult to compute,
P but we can approximate it
by introducing the probability p(Score|z 0 ) = n1 xi p(Score|xi , z 0 ) that the team
scores from a location z 0 on the field, which is the marginal probability of scoring
over players. We will use this approximation in Section 5.
For now, we can use p(Score|z 0 ) to define our scoring map. A scoring map
provides the probability that a team will score from a location z 0 for every
location z 0 on the field. As shown in Figure 1 (right), the probability of scoring
is high when the disc is close to the opponent’s endzone, and low when the disc
is far away. From Figure 1 (right) we see it is also advantageous to have the disc
in the middle of the field. Ultimate experience suggests that such an increase in
scoring probability exists because more in-bounds playing field is accessible with
short throws.
To foreshadow, we can use the completion and scoring maps in conjunction
to better understand ultimate. We will use it recommend where to throw the
disc, how to game-plan for high wind situations, and how to make defensive
adjustments. First however, we use data and nearest neighbor methods to show
that our model maps reflect existing ultimate beliefs.
4 Data maps
While using simplified models to construct completion and scoring maps (mix-
tures of Gaussians[3] are used in Figure 1) to depict belief about probabilities
in ultimate, we want to verify their validity empirically. We collected data using
the UltiApps tracking application[4] based on 2012 film of the Nexgen Tour[5], a
team of college-level all-stars who bus from city to city to play the best club teams
around the United States. The application stores information in a database with
tables for games, teams, points, possessions, and plays, recording the player name
(on offense and defense) and location of each throw. We collected data from 13
games, 10 of which were included in our analysis (the other 3 had coding errors).
The 10 games included 237 points, 501 possessions, and 3195 throws. We extract
relevant information into a single throws table. The throws table contains IDs of
the thrower, receiver and defenders, their locations, the outcome of the throw,
indices for possession, point, and game, all ordered sequentially in time.
From the throws table, we produce empirical completion and scoring maps.
We do this using k-nearest neighbors[6], with k=100. Then for any location z,
we find the nearest neighbors and average the probability of scoring from those
k positions to get our estimate. Figure 2 shows the results for Nexgen and their
opponents. Comparing Figure 1 (right) with Figure 2, we see that the empirical
scoring maps show lower scoring probabilities across the pitch, though as our
1.0
0.8
Own endzone
0.6
0.4
0.2
0.0
Sideline
1.0
p(home_scores), possession
0.8
Own endzone
0.6
0.4
0.2
0.0
Sideline
Fig. 2: Empirical scoring maps for Nexgen (top) and opponents (bottom).
p(opp_scores), possession
model predicted, the proximity to the scoring endzone does improve the chances
substantially.
5 Applications
Completion and scoring maps can be used directly as aids for players, but they
also have other applications. First we show how maps can determine throw
choice. Next, we apply throw choice to show how maps can be used to change
offensive strategy given external factors. Finally, we show how defensive ability
to manipulate the completion map could guide defensive strategy.
Throw choice The best throw a player can make is the one that maximizes the
team’s probability of scoring the point. Note that the probability of scoring the
point is distinct from the probability of scoring the possession. In this section
we relate the two using completion and scoring maps to look at the expected
point-scoring outcomes by throw choice.
Recall from Equation 1 that we get the probability of scoring from considering
all possible paths given (x0 , z 0 ). Now we can consider the probability of scoring
based on where the player chooses to throw. The probability of scoring with a
throw to (x1 , z 1 ) is given by the probability of completing the first throw times
0.05 0.3 0.3
0.20
60 0.15 60 0.00
0.2 0.2
0.10 −0.05
50 50
0.1 0.1
0.05
−0.15
−0.20
−0.05
−0.10
Sideline
Sideline
40 40
0.00
−0.1
0.0 0.0
0
30 30
−0.1 −0.05 −0.1
0.10
20 20
0.15 −0.15
−0.
−0.2 −0.2
0.20
10
● ●
10 10
−0−0.2
0.25
15
0.000.05
.20 5
−0.3 −0.3
−
−0.
10 20 30 10 20 30
E(throw (handler)) E(throw (cutter))
Fig. 3: Expected score maps for handlers (left) and cutters/windy conditions
(right), i.e.. lower probability of completion on the right (lowered by 20 percent).
Note the best decisions are a short throw (left) but a long throw/huck (right).
the probability of scoring from the receiver location:
p(Score|(x0 , z 0 ), (x1 , z 1 )) = p(x0 , z 0 , x1 , z 1 )p(Score|x1 , z 1 )
≈ p(x0 , z 0 , x1 , z 1 )p(Score|z 1 )
This approximation is assuming that the particular receiver of the first throw x1
does not affect the scoring probability. Using this approximation, we can use the
completion map to get p(x0 , z 0 , x1 , z 1 ) and the scoring map to get p(Score|z 1 ). By
decomposing the probability, we now have an approximation to the probability
of scoring given our throw choice.
To find the (approximately) optimal throw, however, we should consider the
expected value given throw choice, not the probability of scoring the possession.
The expected value can be determined by assigning a value of +1 to a score,
and -1 to an opponent score. To simplify further, let us assume that each team
only gets one chance to score. If neither team scores in their first possession,
the expected value is 0. Then we can determine the expected value using the
completion map, the scoring map, and the opponent scoring map. Figure 3 shows
the expected value map given the maps from Figure 1. Then, we can find the
maximum expected value, and instruct the player to throw to that location.
Note that the difference in expected values may seem small–just a fraction
of a score. However, they add up. In the Nexgen games there were an average
of 300 throws per game. If a situation presents itself, for example, 10 times in
a game, choosing an action that makes a difference of 0.1 each time results in
scoring an extra point on average.
Offensive strategy in the wind External factors can govern the completion
and scoring maps. For example, windy conditions lower the probability of com-
pletion, and thus the probability of scoring on a possession. By understanding
or approximating changes in the probabilities, a team can change its mindset.
The map in Figure 3 (right) shows the new expected value map if you lower
the probability of completion in the completion and scoring graphs. While the
original expectation graph in Figure 3 (left) recommended throwing a short pass
(called a reset), in high wind the graph recommends a long pass (called a huck).
Defensive strategy We noted that offensive game-planning, e.g. should a team
throw long, high-risk throws, is affected by knowledge of location-based scoring
probabilities. Similarly, defensive strategies will affect the opponent completion
and scoring probabilities as well. Using scoring maps that take into account de-
fensive positioning, we could identify the minimax outcomes that govern optimal
offensive strategy given defensive strategy. That is, the defense should employ
the strategy that results in the smallest maximum expected value on the map,
and the offense should choose the throw that maximizes the expected value on
the map.
6 Discussion
The analysis presented highlights the capabilities of completion and scoring
maps. Many other uses of the maps and location-based data would be inter-
esting. For one, we can use these maps to characterize players. We can answer
questions such as: how do player completion rates change across the field, and
does the player have weaknesses or strengths in particular regions? We can also
use the expectation maps in conjunction with the throws table to assess how
much “value” each player contributes to the team by summing up the change in
expected values for plays in which the player was involved. Furthermore, we can
perform visual comparative analyses between teams (or players). Subtracting
the scoring maps from one another (or a baseline) can help identify regions of
relative weakness; the empirical comparative scoring map (difference in maps in
Figure 2) is shown in Figure 4.
While location-based tracking adds a visual and predictive component that
helps describe optimal ultimate play, it does not provide other pertinent infor-
mation that teams and players must address when making decisions on the field.
For example, the location of players not involved the movement of the disc affect
the choices made. An alternative analysis could track not only the disc move-
ment but all 14 player’s movements. Also, while our analysis uses the sequence
of throws (to determine possessions and scores), the analysis is atemporal. Many
throws are relatively easy off of disc movement because the defenese is out of
0.3
0.2
Own endzone
0.1
0.0
−0.1
−0.2
−0.3
Sideline
Fig. 4: Empirical scoring map difference (purple indicates Nexgen outperforming
opponents).
home minus opp p(scores), possession
position, and an atemporal model does not capture these elements of the game.
Another important factor, weather condition, goes unmodeled. Incorporating
these into our models would help refine our analysis and additional insight into
ultimate strategy. Finally, the number of throws available for analysis will al-
ways be relatively small, particularly against uncommon strategies or in unusual
conditions. Developing strategies and assessing individual ability in the face of
limited data will be challenging and should be considered in ultimate analyses.
Acknowledgments
We would like to acknowledge Ultiworld and UltiApps for their continued sup-
port and development of ultimate statistics.
References
1. J. Albert, “An introduction to sabermetrics,” Bowling Green State University
(http://www-math. bgsu. edu/˜ albert/papers/saber. html), 1997.
2. K. Goldsberry, “Courtvision: New visual and spatial analytics for the nba,” MIT
Sloan Sports Analytics Conference, 2012.
3. B. S. Everitt and D. J. Hand, “Finite mixture distributions,” Monographs on Applied
Probability and Statistics, London: Chapman and Hall, 1981, vol. 1, 1981.
4. “UltiApps stat tracker.” http://ultiapps.com/. Accessed: 2013-06-28.
5. “NexGen.” http://nexgentour.com/. Accessed: 2013-06-28.
6. B. V. Dasarathy, “Nearest neighbor ({NN}) norms:{NN} pattern classification tech-
niques,” 1991.