=Paper=
{{Paper
|id=Vol-2068/uistda3
|storemode=property
|title=SportSense: User Interface for Sketch-Based Spatio-Temporal Team Sports Video Scene Retrieval
|pdfUrl=https://ceur-ws.org/Vol-2068/uistda3.pdf
|volume=Vol-2068
|authors=Lukas Probst,Ihab Al Kabary,Rufus Lobo,Fabian Rauschenbach,Heiko Schuldt,Philipp Seidenschwarz,Martin Rumo
|dblpUrl=https://dblp.org/rec/conf/iui/ProbstKLRSSR18
}}
==SportSense: User Interface for Sketch-Based Spatio-Temporal Team Sports Video Scene Retrieval==
https://ceur-ws.org/Vol-2068/uistda3.pdf
SportSense: User Interface for Sketch-Based
Spatio-Temporal Team Sports Video Scene Retrieval
Lukas Probst, Ihab Al Kabary, Rufus Lobo, Philipp Seidenschwarz, Martin Rumo
Fabian Rauschenbach, Heiko Schuldt Centre for Technologies in Sports and Medicine
Dept. of Mathematics and Computer Science Bern University of Applied Sciences, Switzerland
University of Basel, Switzerland firstname.lastname@bfh.ch
firstname.lastname@unibas.ch
ABSTRACT packages such as Sportscode1 , myDartfish2 or Viz Libero3
In the last years, various sports have seen significant efforts in help segmenting videos through tagging and help enhancing
collecting large volumes of video, mainly for analytical pur- video content through manually added schematic video over-
poses. These videos are tagged with events that include spatial lays. Still, the information is not obtained from a systematic
and temporal information. In order to avoid that game analysts analysis but stays anecdotal.
have to manually analyze large video collections, appropri-
Yet, in order to facilitate the analysis of large video collec-
ate user interfaces are needed for finding characteristic video
tions, appropriate user interfaces are needed for advanced
scenes. S PORT S ENSE is a novel sketch-based video retrieval
video retrieval to find characteristic video scenes. In this pa-
system tailored to the needs of sports video analysts which
per, we present S PORT S ENSE, a novel sketch-based video
include spatio-temporal queries such as player movements,
scene retrieval system tailored to the needs of sports video
ball trajectories, or interactions between players. We present
analysts which include spatio-temporal queries such as the
the user interface of S PORT S ENSE that allows users to draw
movement of a player, the trajectory of the ball, and/or inter-
sketches of spatio-temporal events on the field and that visual-
actions between players. Given that tracking and event data
izes the retrieved video scenes. We introduce the data model
will become a commodity, an information retrieval tool such
of S PORT S ENSE, show the sketch-based spatio-temporal re-
as S PORT S ENSE will help to bridge the gap between videos
trieval on the annotated videos, and present a qualitative eval-
and events, and also between anecdotal information / simple
uation exhibiting the effectiveness of the S PORT S ENSE UI.
statistics and more sophisticated analysis results.
ACM Classification Keywords The contribution of this paper is threefold: (i) we introduce
H.3.3. Information Storage and Retrieval: Information Search a generic model for spatio-temporal tracking data, events,
and Retrieval; H.5.2. Information Interfaces and Presentation statistics, and match metadata, independent of concrete sports;
(e.g. HCI): User Interfaces (ii) we show the user interfaces of S PORT S ENSE for sketch-
based video scene retrieval in football and ice hockey; and
(iii) we present the results of user studies showing the effec-
Author Keywords tiveness and user friendliness of S PORT S ENSE.
Sketch Interfaces, Spatio-Temporal Video Retrieval, Sports
The remainder of the paper is structured as follows: Section 2
introduces the architecture of S PORT S ENSE and Section 3 its
INTRODUCTION generic data schema for video annotations. Section 4 sum-
The analysis of videos is more and more becoming important marizes the spatio-temporal query types of S PORT S ENSE and
in a large variety of team sports. For this, videos on previous Section 5 presents the UIs for football and ice hockey. The
matches are tagged, either manually or automatically, with results of our usability study are discussed in Section 6. Sec-
events that are characterized by spatial (relative to the field) tion 7 presents related work and Section 8 concludes.
and/or temporal information. Conventionally, analyzing tac-
tical elements of teams in sports videos is a tedious, manual
activity of game analysts with the objective of understanding ARCHITECTURE
the strengths and weaknesses of the next opponent for optimal In this section, we present the architecture of S PORT S ENSE
preparation and for taking specific tactical decisions. Software (see Figure 1) which consists of three components: a Web
client, a REST proxy, and a database backend.
The database stores all available tracking data (e.g., positions
of a player), events (e.g., passes) and statistics (e.g., pass
statistics). In addition, it stores metadata for all matches.
1 https://www.hudl.com/elite/sportscode
2 http://www.dartfish.com/Products
©2018. Copyright for the individual papers remains with the authors.
3 http://www.vizrt.com/products/viz_libero
Copying permitted for private and academic purposes.
UISTDA ’18, March 11, 2018, Tokyo, Japan
HTTP request DB query
Web REST { "type" : ,
" matchId ": ,
Client Proxy DB
User draws/clicks HTTP response DB result
"ts ": ,
" videoTs ": ,
" xyCoords ": [[,], ... ,[,]],
Figure 1. SportSense Architecture4 " zCoords ": [, ... ,],
" playerIds ": [, ... ,],
The user can use the sketch-based interface of the Web client " teamIds ": [, ... ,],
" additionalInfo ": {
for different types of spatio-temporal queries to the database. : ,
Moreover, the Web client visualizes the query results (events ...
and associated video scenes) for the user in an intuitive way. : } }
Listing 1. Generic Tracking Data and Event Schema
Communication between the Web client and the database is
handled via a REST proxy which allows to query the database { "type" : " position ",
using a restful API. It receives HTTP requests from the Web " matchId ": "510350" ,
"ts ": 1405278313179 ,
client and transforms them into database queries. Subse- " videoTs ": 313 ,
quently, on receiving the query results from the database, it " xyCoords ": [[2.367 ,17.739]] ,
" zCoords ": [0.0] ,
returns HTTP responses to the Web client. " playerIds ": [" A8"],
" teamIds ": ["A"],
DATABASE " additionalInfo ": {
"v": [0.409 , -1.136 ,0.0] ,
Spatial databases [4] focus on storing and querying spatial "vabs ": 1.207 } }
data like GPS tracks. They provide two or three-dimensional Listing 2. Sample Tracking Data. Player A8 was located at
spatial index structures and support spatial query operators h2.367, 17.739i, had a velocity of 0.409 m/s in x, −1.136 m/s in y and 0.0 m/s
like distance, intersection, or containment. in z direction and an absolute velocity of 1.207 m/s.
The database component of S PORT S ENSE is used to store tem- { "type" : " passEvent ",
" matchId ": "510350" ,
poral and spatial information, especially tracking and event "ts ": 1405279175498 ,
data. While every tracking data item contains exactly one " videoTs ": 1175 ,
position, an event data item may even contain multiple posi- " xyCoords ": [[ -6.041 ,7.888] ,[4.816 ,3.682]] ,
" zCoords ": [0.0 ,0.0] ,
tions. For instance, a data item that reflects a pass contains " playerIds ": [" A3","A5"],
two positions – the start and end location of the pass – which " teamIds ": ["A"],
" additionalInfo ": {
form a line. As we are interested in events that took place in " packing ": 3.0 } }
a specific region, S PORT S ENSE uses a database with support
Listing 3. Sample Event. Successful pass from player A3 located at
for two-dimensional containment queries. h−6.041, 7.888i to player A5 located at h4.816, 3.682i with a packing
value of 3.
The objective of S PORT S ENSE is not only to enable a user to
retrieve video scenes of a single match, but also to retrieve
video scenes of hundreds of matches (e.g., from several sea- the planar tracking position or the position(s) where the event
sons) for analytical purposes. Therefore, we need to make took place. The zCoords array contains the corresponding
sure that the database scales with the volume of tracking and z coordinates. The playerIds and teamIds array contain the
event data. Based on comparisons of spatial database systems involved players and teams.
by Schmid et al. [7] and Agarwal et al. [1], we have decided to There are two varying factors in the mandatory part of the
use MongoDB5 because its feature set supports containment schema. The first factor is the number of positions. The
queries with arbitrary polygons and it exhibits a satisfactory xyCoords array can contain arbitrarily many positions. Every
performance and scaling behavior. MongoDB is a document position in this array is a so called legacy coordinate pair6 ,
database belonging to the NoSQL database family. It stores that is, a point in a planar two-dimensional coordinate sys-
collections of documents which are binary JSON objects. In tem. A containment query returns an event if at least one7 of
contrast to relational databases, MongoDB does not enforce its positions is contained in the specified region. In order to
a static schema. Nevertheless, we define three schemata for accelerate the containment queries, we leverage MongoDB’s
S PORT S ENSE as this simplifies spatio-temporal queries. two-dimensional index structure. Since MongoDB does not
support three-dimensional indexes, the z coordinate is out-
Tracking Data and Event Schema sourced to the zCoords array and thus separated from the x
First, we define a generic schema for the tracking and event and y coordinates. However, this is not an issue since the z co-
data. This schema is presented in Listing 1. The type, matchId, ordinate is not required for S PORT S ENSE’s queries as the user
ts, videoTs, xyCoords, zCoords, playerIds and teamIds fields interfaces support only two-dimensional sketches. The second
are mandatory. The type specifies the type of the event or track- factor is the varying number of player and team identifiers in
ing data, resp. The matchId assigns the data item to a match the playerIds and teamIds array. For instance, a tracking data
and ts denotes the time (in ms) at which the event took place or item involves exactly one player and one team (see Listing 2),
a tracking position has been captured. Additionally, videoTs a pass event involves two players from a single team (see
specifies the video offset (in s). The xyCoords array contains 6 https://docs.mongodb.com/manual/geospatial-queries/
4 User icon made by Freepik from https://www.flaticon.com 7 It is possible to specify spatial constraints for a specific position in
5 https://www.mongodb.com the array by means of leveraging the dot operator (e.g., xyCoords.2).
{ "type" : , { " matchId ": ,
" matchId ": , " sport ": ,
"ts": , " fieldSize ": [,],
" videoTs ": , "date ": ,
" playerIds ": [, ... ,], " competition ": ,
" teamIds ": [, ... ,], " venue ": ,
" additionalInfo ": { " homeTeamId ": ,
: , " awayTeamId ": ,
... " homeTeamPlayerIds ": [, ...] ,
: } } " awayTeamPlayerIds ": [, ...] ,
" homeTeamName ": ,
Listing 4. Generic Statistics Schema " awayTeamName ": ,
" homeTeamPlayerNames ": [, ...] ,
{ "type" : " passStatistic ",
" awayTeamPlayerNames ": [, ...] ,
" matchId ": "510350" ,
" videoPath ": ,
"ts": 1405279825763 ,
" homeTeamColor ": ,
" videoTs ": 2425 ,
" awayTeamColor ": }
" playerIds ": [" B8"],
" teamIds ": ["B"], Listing 6. Generic Match Metadata Schema
" additionalInfo ": {
" numPasses " : 12, { " matchId ": "510350" ,
" numInterceptions " : 3, " sport ": " football ",
" avgPacking " : -0.34 } } " fieldSize ": [110.0 ,75.0] ,
Listing 5. Sample Statistics. Pass statistics for player B2. "date ": "2014 -07 -13 T19 :00:00 Z",
" competition ": "FIFA World Cup 2014" ,
" venue ": " Maracana ",
Listing 3), an interception event involves two players and two " homeTeamId ": "A",
teams and an offside trap event contains only a team but no " awayTeamId ": "B",
specific player. Although, the number of positions, players, " homeTeamPlayerIds ": [" A1", ...] ,
" awayTeamPlayerIds ": [" B1", ...] ,
and teams depend on the type their structure remains static. " homeTeamName ": " Germany ",
" awayTeamName ": " Argentina ",
The additionalInfo field enables specifying more event and " homeTeamPlayerNames ": [" Manuel Neuer ", ...] ,
tracking data type specific information. For instance, the " awayTeamPlayerNames ": [" Sergio Romero ", ...] ,
" videoPath ": "./ path/to/ video .mp4",
additionalInfo field for player position tracking data con- " homeTeamColor ": " white ",
tains the velocity of the player (see Listing 2) while the " awayTeamColor ": "blue" }
additionalInfo field for pass events contains the packing value Listing 7. Sample Match Metadata. FIFA World Cup final 2014.
which denotes the number of players of the opposing team
that are outplayed by this pass (see Listing 3). information. The matchId is the identifier of the match and
referred in the tracking, event, and statistics data items. The
While the fixed structure of the mandatory information enables sport field specifies the discipline and fieldSize defines the size
consistent accesses to common information (e.g., retrieve all of the sports field. The subsequent fields inform when (date),
events which are located in a specific region of the field by in which context (competition) and where (venue) the match
means of a single database query) and the definition of indexes took place. The videoPath specifies the path where a video
for efficient access, the additionalInfo field makes the schema of the match can be found. The homeTeamColor and the
flexible and thus extensible towards arbitrary new event types. awayTeamColor specify colors which can be used for visual-
izations. The remaining fields can be used to map the player
Statistics Schema
and team identifiers used in the tracking, event, and statistics
Second, we define a generic schema for storing statistics (see data items to player and team names. For instance, in List-
Listing 4). The sole but crucial difference to the tracking data ing 7 team identifier “A” is mapped to “Germany” and player
and event schema is that the statistics schema has no xyCoords identifier “A1” is mapped to “Manuel Neuer”.
and zCoords field, i.e., contains no position information, as a
statistics data item is not associated with certain positions on SPATIO-TEMPORAL QUERIES
the field but corresponds to aggregated values. In this section, we present four types of sketch-based spatio-
The actual statistics values as well as other statistic type spe- temporal queries for retrieving team sports video scenes. All
cific information (if necessary also positions) are stored in the query types have in common that they are not tailored to one
additionalInfo field. For instance, the additionalInfo field of sport discipline, but can be applied to different team sports.
pass statistics data items contains the number of passes and
interceptions as well as the average packing of a certain player Region Queries
(see Listing 5) or team (if playerIds is empty). The most straightforward query type is to retrieve all events
that are located in a specific region on the field [2]. In football,
Match Metadata Schema this query type can be used for instance if the user is interested
Third, we define a generic match metadata schema depicted in in all shots on goal that originated from outside of the penalty
Listing 6 and exemplified in Listing 7. This schema is used to box. The region for such a query can be defined by sketching
store information about matches. There is exactly one match a free form on the field (see Figure 2).
metadata item for every match.
The results of such a region query can be further refined by
In contrast to the other two schemata, the generic match meta- specifying the event type, the player, and/or the team. For
data schema has a strict structure and no field for flexible spatio-temporal queries, also a time interval is specified.
the last event and retrieve the preceding events by expanding
all results [3]. This can be continued by repeating the expand
action. To shrink the set of event cascades, the user can define
additional regions prior to every expand action (see Figure 5).
USER INTERFACES
The Web client of S PORT S ENSE and thus its user interface is
different for every sport discipline. Currently, we provide an
Figure 2. Sample Region Query Sketch in Football UI for football and one for ice hockey.
10 20 30 40 50 40 30 20 10
SportSense-Football
S PORT S ENSE -F OOTBALL is the football UI of S PORT S ENSE.
This Web client is implemented in TypeScript and uses
10 20 30 40 50 40 30 20 10 Bootstrap8 , Bootstrap Multiselect9 , jQuery10 , jQuery UI11 ,
DataTables12 and heatmap.js13 . Figure 6 shows a screenshot
Figure 3. Sample Motion Query Sketch in American Football
captured after a successful motion query. The user interface
can be partitioned into four areas: The navigation, the field,
the result list, and the video pane.
The navigation on the left hand side contains various drop-
downs and sliders grouped into four categories and adapts to
the current query type. The drop-downs and sliders can be
Figure 4. Sample Forward Event Cascade Query Sketch in Ice Hockey
used to set filters, to change between the query types, to set
query type specific parameters, and to induce queries.
The football field at the bottom serves two purposes. In draw-
ing mode, the user can sketch on the field in order to specify a
region for a query. S PORT S ENSE -F OOTBALL supports draw-
ing rectangles, circles, and free form sketches. Moreover,
S PORT S ENSE -F OOTBALL allows to mirror the sketch for re-
Figure 5. Sample Reverse Event Cascade Query Sketch in Handball gion and motion queries to handle different playing directions.
Besides sketching, the field is used to visualize the query re-
Motion Queries sults. In selection mode, the user can select a result of a region
Another query type is to retrieve video scenes in which the or cascade query by clicking on the corresponding visualiza-
movement of the ball or a player follows a specific path (i.e., is tion. The user can switch between the modes by clicking on
within a certain area) [2]. In American football, this query the buttons above the field.
type can be used for instance to retrieve a certain play that
Alternatively, the user can select a result in the result list at
has led to a touchdown after a long run. Such a query can be
the right hand side. The result list enables the user to sort the
defined by drawing the movement of the ball, i.e., its motion
results by different criteria (e.g., by the matchId, the duration
path, on the field (see Figure 3).
and the covered distance in case of a motion query).
Forward Event Cascade Queries Once a user has selected a result on the field or in the result
Assume that a user wants to retrieve a video scene based on a list, the video pane at the top starts playing the corresponding
specific sequence of events she remembers, such as a sequence scene. The slider below the video can be used to specify how
of passes in the bottom right part of an ice hockey field. In this much ahead of the result the video should start.
case, using the region query would not be sufficient. Instead,
Besides the four spatio-temporal query types introduced be-
the user has to specify a query that includes the temporal
fore, S PORT S ENSE -F OOTBALL supports two purely temporal
sequence of events in the form of a forward event cascade [2].
queries. More precisely, the user can retrieve the heatmap
In order to specify such a query, the user can subsequently
for a certain player and interval (e.g., for player A3 between
draw a region for every event in the sequence (see Figure 4).
minute 10 and 25) as well as the pass statistics at a certain
time (e.g., at minute 42). The time boundaries for the heatmap
Reverse Event Cascade Queries
interval and the point of time for the pass statistics can be
Assume that a user wants to analyze which sequence of events
specified by means of a slider.
has led to a specific event she remembers. For instance, in
handball a coach might want to analyze which pass patterns 8 http://getbootstrap.com
have led to a goal from the right back area. In such cases, 9 https://github.com/davidstutz/bootstrap-multiselect
the user does usually not know the full sequence of events 10 https://jquery.com
in advance. Instead, the event cascade has to be built incre- 11 https://jqueryui.com
mentally and interactively in reverse order. To perform such a 12 https://datatables.net
reverse event cascade query, the user can draw the region of 13 https://www.patrick-wied.at/static/heatmapjs
Figure 6. Mirrored Motion Query in SportSense-Football Figure 7. Forward Event Cascade Query in SportSense-Icehockey
SportSense-Icehockey To evaluate a user interface, each participant had to rate the
The ice hockey Web client of S PORT S ENSE, called different query types as well as the overall application w.r.t.
S PORT S ENSE -I CEHOCKEY, is implemented in TypeScript several questions on an evaluation sheet (see left part of Fig-
and uses Bootstrap, Bootstrap Select14 , fabric.js15 , jQuery and ure 8). The rating schema we have used was: 1 for “very
video.js16 . A screenshot showing the result of a successful bad”, 2 for “bad”, 3 for “fair”, 4 for “good” and 5 for “very
forward event cascade query is given in Figure 7. good”. Alternatively, a participant has been able to abstain
from answering a question. Unanswered questions are ig-
The user interface is partitioned into five areas. Three of nored in the result interpretation. Moreover, the participants
them – the field, the result list, and the video pane – are very have been able to give additional feedback as free text. The
similar to those in S PORT S ENSE -F OOTBALL. However, so evaluation sheet for S PORT S ENSE -I CEHOCKEY has been a
far, S PORT S ENSE -I CEHOCKEY lacks a few features that have subset of the evaluation sheet for S PORT S ENSE -F OOTBALL
already been implemented in S PORT S ENSE -F OOTBALL. The as S PORT S ENSE -I CEHOCKEY only supports a subset of the
field does not yet support free sketches and the mirror function, query types of S PORT S ENSE -F OOTBALL.
the result list cannot be sorted and there is no slide to control
the video offset. Moreover, since tracking data, statistics and For every spatio-temporal query type, all participants were
heatmaps for ice hockey are not yet available in both quantity given a task (such as, for instance, retrieve all video scenes
and detail as in football, motion, heatmap and statistics queries with a goal shot by a certain player) they had to com-
are not supported but will eventually be added. plete before answering the corresponding questions. The
tasks have been the same for all participants but different
In contrast to a large navigation area with many drop-downs in S PORT S ENSE -F OOTBALL and S PORT S ENSE -I CEHOCKEY.
and sliders as in S PORT S ENSE -F OOTBALL, S PORT S ENSE -
I CEHOCKEY has a dedicated filter area at the top right. The Figure 8 shows the average rating as well as the standard
actual navigation for specifying the query type, entering the deviation for all answers. The majority of the average ratings
drawing mode and inducing queries is between the video pane is between 4 (good) and 5 (very good). This confirms the high
and the field area. This navigation contains much less elements level of user satisfaction in both interfaces.
and hides elements that are not required for the current query The two outliers for S PORT S ENSE -I CEHOCKEY – the ease
type. For instance, the drop-down for switching between of use (2.917) and the spatial aspects (3.5) for the forward
the forward and the reverse event cascade query type is only cascade – can be explained by the fact that the task for the
visible if the event cascade box is ticked. If further input is forward event cascade query type has been quite hard. The
required (e.g., to select the types for expanding a reverse event participants had to sketch very narrow and nearby regions
cascade query) a pop-up is shown to the user. for an event sequence – while these regions have not been
allowed to overlap17 . We suppose that this is also the reason
EVALUATION why S PORT S ENSE -I CEHOCKEY has a lower overall ease of
We have run separate user studies for S PORT S ENSE - use rating (3.75) than S PORT S ENSE -F OOTBALL (4.0).
F OOTBALL and S PORT S ENSE -I CEHOCKEY to evaluate the Although the overall ease of use rating in the S PORT S ENSE -
usability of both user interfaces. The group of participants in F OOTBALL user study is good at large, the fact that the ease
both user studies has been different but not completely disjoint. of use ratings for the spatio-temporal query types (4.0, 3.8,
S PORT S ENSE -F OOTBALL has been evaluated by 20 partici- 3.3 and 3.95) and the overall intuitiveness (3.55) are among
pants and S PORT S ENSE -I CEHOCKEY has been evaluated by the lowest results in the S PORT S ENSE -F OOTBALL user study
twelve participants. In both user studies, the participants have indicates that there is still potential for further improvement.
been guided by the developer of the system. This is also reflected in the free text feedback provided by
14 https://github.com/silviomoreto/bootstrap-select
the participants. While in the feedback comments on the
15 http://fabricjs.com
S PORT S ENSE -F OOTBALL user study 45 % of the participants
16 http://videojs.com 17 We have fixed this issue already in the latest version.
Question Query Type User Ratings for SportSense-Football User Ratings for SportSense-Icehockey
Accuracy of the result Region 4.650 4.286
Forward Cascade 4.550 4.200
Reverse Cascade 4.368 4.333
Motion 4.500
Heatmap 4.722
Pass Statistic 4.789
Ease of use of tool Region 4.000 4.500
Forward Cascade 3.800 2.917
Reverse Cascade 3.300 4.333
Motion 3.950
Heatmap 4.526
Pass Statistic 4.400
How well can you express Region 4.400 4.083
the spatial aspect of the Forward Cascade 4.100 3.500
query?
Reverse Cascade 4.050 4.364
Motion 4.263
How well does the result Region 4.600 4.250
visualization on the field Forward Cascade 4.350 4.083
represent the result?
Reverse Cascade 4.150 4.083
Motion 4.350
Effectiveness of result Heatmap 4.778
visualization Pass Statistic 4.105
Aesthetics of the design Overall Application 3.850 4.167
Ease of use Overall Application 4.000 3.750
Intuitiveness Overall Application 3.550 4.083
1 2 3 4 5 1 2 3 4 5
Figure 8. Evaluation Results. The graph shows the average and the standard deviation.
criticized the intuitiveness or ease of use, only 25 % of the client that combines the best of S PORT S ENSE -F OOTBALL and
participants did so in the S PORT S ENSE -I CEHOCKEY user S PORT S ENSE -I CEHOCKEY and that will serve as a user inter-
study. Moreover, 30 % of the participants raised issues with face for arbitrary team sports. Moreover, we aim to introduce
the scrolling in the navigation of S PORT S ENSE -F OOTBALL18 . new sketch-based spatio-temporal query types for retrieving
In consequence, we deduce that S PORT S ENSE users prefer a video scenes based on (multi-player) events, trajectories, and
more lightweight navigation concept like in S PORT S ENSE - specific team behavior patterns (i.e., the team’s tactics).
I CEHOCKEY. For that reason, we plan to implement a more
lightweight navigation in our future user interfaces. ACKNOWLEDGMENTS
This work has been partly supported by the Hasler Foundation
RELATED WORK (project StreamTeam, contract no. 16074).
Systems for visually analyzing matches in team sports rely on
event data that is either provided manually (e.g., by companies REFERENCES
like Opta [5]) or automatically. The latter is done by extracting 1. S. Agarwal and KS Rajan. 2015. Performance Analysis
raw position data from sensors deployed on the players [10] or of MongoDB Vs. PostGIS/PostGreSQL Databases For
from videos and analyzing theses position data, i.e., detecting Line Intersection and Point Containment Spatial Queries.
events and calculating statistics, offline or in real-time [6]. In Proc. of FOSS4G.
The work of Stein et al. [9] focuses on the analysis of player 2. I. Al Kabary and H. Schuldt. 2013. Towards Sketch-
and ball motion on the field and the visualization of the analy- based Motion Queries in Sports Videos. In Proc. of ISM.
sis results, as an overlay to the video of the match. Chalkboard- 3. I. Al Kabary and H. Schuldt. 2014. Enhancing Sketch-
ing [8] is a spatio-temporal approach that focuses on retrieving based Sport Video Retrieval by Suggesting Relevant
particular plays in sports. The system offers the following Motion Paths. In Proc. of SIGIR.
query types: (i) exemplar-based queries, (ii) manipulated (by a
user) exemplar-based queries, and (iii) drawing-based queries. 4. R. Güting. 1994. An Introduction to Spatial Database
Since Chalkboarding does not rely on explicit event data, event Systems. VLDB Journal 3, 4 (1994).
cascades like in S PORT S ENSE cannot be provided. 5. Opta Sports. 2017. The Data Collection Process.
http://www.optasports.com/about/how-we-do-it/the-data-
CONCLUSION & FUTURE WORK
collection-process.aspx. (2017).
In this paper, we have introduced S PORT S ENSE, a system for
retrieving video scenes of team sports matches by means of dif- 6. L. Probst, et al. 2017. Demo: Real-Time Football
ferent types of spatio-temporal queries. Moreover, we have de- Analysis with StreamTeam. In Proc. of DEBS.
fined generic schemata for storing tracking data, events, statis- 7. S. Schmid, et al. 2015. Performance investigation of
tics, and match metadata of arbitrary team sports. We have selected SQL and NoSQL databases. In Proc. of AGILE.
presented two user interfaces, S PORT S ENSE -F OOTBALL and
S PORT S ENSE -I CEHOCKEY, tailored to particular sports, that 8. L. Sha, et al. 2016. Chalkboarding: A New
enable the user to intuitively define spatio-temporal queries to Spatiotemporal Query Paradigm for Sports Play Retrieval.
retrieve video scenes by drawing sketches on the field. Our In Proc. of IUI.
user studies have confirmed the effectiveness of the sketch- 9. M. Stein, et al. 2018. Bring It to the Pitch: Combining
based retrieval approach in sports and a high level of user Video and Movement Data to Enhance Team Sport
satisfaction in both user interfaces. Analysis. Trans. Vis. Comput. Graphics 24, 1 (2018).
In our future work, we plan to leverage the insights acquired 10. A. Stelzer, et al. 2004. Concept and Application of LPM –
from the evaluation to develop a unified S PORT S ENSE Web A Novel 3-D Local Position Measurement System. Trans.
18 The UI has been evaluated with a lower resolution than in Figure 6. Microw. Theory Techn. 52, 12 (2004).