The spatial configuration of sports teams, such as soccer matches, indicates their tactic. The specification of team tactics might be either ofensive or defensive. We propose a method using Meta Interpretive Learning (MIL) to generate rules that are learned from a video to explain the strategy of a football game. We first track the players to estimate their position and estimate the team's formation. For the purpose of classifying players as defender, midfielder, or attacker, our method combines k-means and OPTIC clustering. We measure the dynamic strategy within the time series by generating background knowledge, then a new rule extracts to explain team strategy regarding the current state. Finally, in order to determine the accuracy using MIL, our experiments compare our approach with a long term short term memory (LSTM) model. In contrast, our research demonstrates the superiority of a MIL system over a deep learning model with a small dataset.
Numerous professional sports clubs have recently embraced camera-based monitoring
technology that frequently records the whereabouts of both players and the ball. However, they
rarely use the crucial information that is concealed in these performance data while making
decisions.The computational methods required to fully analyse these data [
Gameplay configuration over a given period is used to predict the match using time series
analysis [
In this paper, we propose a method to automatically learn the rules from the spatial configuration of players, extracted from a soccer video. We first employ a deep learning model to detect players and statistical method to determine team formation from noisy data. Then we use meta interpretive learning (MIL) techniques with Metagol as the rule learning engine. More precisely, we are interested in identifying and analyzing team behavior that reveals the team’s tactics. Our proposed method must take into account player distance and team formation. Additionally, several players may be involved in a single action in team sports like football and basketball. We need to analyze all players in order to correctly recognize the action[5]. To address this issue, we employ ILP approaches that make use of Prolog programming, which can easily handle relational data.
Although there hasn’t been much research on game analysis utilizing logic and ILP methods,
other techniques like deep learning and statistical methods produce respectable outcomes.
Logic and semantic methods: Vercruysse et al. [6]. proposed a qualitative spatial reasoning
approach by investigating the information between players and the dynamic of the game. Their
method is to predict if a player passes the ball to the front or to the other players. Automatically
discovering the soccer match data to recognize the pattern of ofensive by Van Haaren and
others[
Statistical and deep learning methods: Most recent research using the LSTM model shows the feasibility of prediction using deep learning. Sports match prediction model [8] proposed a method to combine the LSTM model with the attention mechanism and put forward an AS-LSTM model for predicting match results. [9] examined the playing tactic on goal scoring with assessing opponent interaction. Their approach involved analysing opponent interactions in Norwegian elite men’s soccer using a case-control design to analyse the efects of playing strategies, counterattack versus extravagant assault, on the likelihood of goal scoring. Other approaches with spatial analysis define how a team plays defensively or ofensively. [ 10] proposed a dynamic analysis of team strategy, their method classifies team formation and detects major tactical changes during the course of a match. Their method relies on the spatial shape of teams’ players. Collective movement analysis [11] examines the players’ motion patterns and the underlying coordination among them, their method provides a comprehension of the collective tactics that contribute to team efectiveness. [ 11] reviewed the influence of player position to understand their tactical performance. They measured players’ distances during a match by tracking their location and measuring the time players were closer than a threshold distance. They also investigated the distance of players depending on their roles. K. Kim et al. [12] provided a novel approach to predict play evolution by extracting the ground level sparse movement of players. Their data utilize each time step and then generate a dense motion field.
This section outlines our approach to player tracking as well as our methods for grouping players and estimating their distances. We utilize this data as the raw data. Then we describe the process by which we create relational data from raw data. From the video provided by the [13], we first identify each player using pre-trained single shot detection (SSD) [ 14] model. Then we fetch players position by the coordinate of bounding box and store them in a data file, frame by frame. Each frame in the data file consist of players position and their team name (e.g teamA or teamB). By calculating the distance to each player, we keep track of the ball’s position to determine which team is in possession. In section 6.1 we explain the usage of the dataset and generating background knowledge.
There is no assurance that to accurately detect objects in videos. There are many factors that can lead to the system misidentifying objects from frame to frame, including an improperly trained convolutional neural network(CNN) model (i.e. A model with insuficient train size), noisy frames, shadows, and brightness variations [15]. Consequently, it is dificult to locate things accurately from videos. To address this issue, we take into account twenty frames of a player’s position and utilize K-means to look at the centroid of all positions. As a result, the valid positions of all players can be calculated on the field. This technique enables the system to function without noise or missing data. In order to identify the centre of positions, we use the k-means[16] algorithm described in equation 1 with one cluster for each participant. Σ=1Σ‖ − ‖2 (1) Where denotes the mean of a player’s positions collected from a set of 20 frames. As a result, there are 10 positions on the pitch, excluding the goalkeeper, which are divided into three groups: defenders, midfielders, and attackers. For this technique, we use the procedure below 2 to determine the clustering structure from ordering points(OPTICS)[17]: core-dist, minPts() = {︃undefined ,
if | ()| < minPts minPts-th smallest distance in (), otherwise (2) Where denotes the distance(radius) of neighbours with a minimum number of points to form a cluster( ). By linearly arranging the points, the method creates an improved ordering of the data, making the physically closest points to one another neighbours. This cluster-ordering comprises data that is comparable to the clusterings created using density-based methods and covering a wide variety of parameter choices. Figure 2 demonstrates how the noisy data is categorized into soccer roles. Every category counts as one of the set positions in the formation, for example, 4 defenders, 3 midfielders and 3 attackers (4-3-3 formation), which means isolating full-back and supporting attackers. The categorization of the data is repeated over the course of the following 20 frames, allowing the system to recognise dynamic formations during the match.
LSTM [18] models have emerged as a pivotal advancement in the field of deep learning and sequential data analysis. LSTMs are characterized by their unique architecture, featuring a network of interconnected memory cells that can store and update information over extended sequences. Unlike conventional RNNs, LSTMs are equipped with gating mechanisms, including input, output, and forget gates, which enable them to selectively retain or discard information at each time step. This gating mechanism enhances their capacity to maintain relevant context and prevent the loss of crucial information, making them particularly well-suited for tasks that involve capturing patterns and relationships over extended temporal horizons. In our experiment, we incorporate LSTM as a point of comparison with our own method because this model takes into consideration the temporal nature of time series data and serves as an efective model for predicting match outcomes.
MIL is a form of ILP [19, 20]. A set of examples and background knowledge made up of a set of Prolog definitions and metarules such that = ∪ are provided to the learner. To produce a hypothesis H such that , |= is the goal. The Prolog meta-interpreter has been modified for the proof [ 21]. The ability to learn recursive algorithms and support for predicate invention are two of MIL’s core characteristics. The former enable the program to break down the repetition of predicates. The latter makes it possible to decrease the textual complexity of generated rule. A MIL system called Metagol [22], can produce rules from examples, background knowledge, and metarules.
We prepare the initial background knowledge (BK) using Prolog and based on a relational approach. Relational learning is a method of MIL which represents the relation between each rule. For example, the rule
(, ) ← (, ), (, ). denotes the tactic of team is if attackers of team are at the position and have possession of the ball . The BK includes a list of facts (, ) such as formation types, team’s opponent, player roles type (i.e. constant, shrink, expand), ball possession and team position in the field. Each predicate demonstrates fact name , relational name and relational settings . The system updates BK for a new rule after the data regenerates, over a 20-frame sequence. The names of the formations (e.g. 4-3-3 or 5-2-3), as well as the players’ distribution in relational mode(i.e. expand, shrink and constant), can be retrieved from the clustered data. We first calculate the variance percentage of each role within 200 frames using equation 3. Then a Python code relatively generates predicates given from each percentage and it saves predicates into a Prolog BK file. The learner in section 7 repeats learning from new BK and it generates new rules for each sequence. The Table 2 shows how BK file updates with new data. The data is sequentially increased by additional predicates like ball possession and team position, see Section 6.2.
= ( 2() − 2(− 1))/ 2() * 100 (3) Where denotes player distance in each role from the frame . Therefore, we compute changes in player distance over a period of time. If a value is , it signifies that the distance between the players hasn’t altered significantly; otherwise, it would therefore be ℎ or . The method is motivated from The influence of player position [ 23] and activity analysis of football players[24]. The method is to take into account the separation between each player in their own role.
Success in soccer has been correlated with the capacity to maintain possession of the ball for extended periods of time [25]. On the other hand, the centroid locations and surface areas of two teams may be used to explain the coordinated flow of ofence and defence at the team level [26]. Thus, we take into account the possession of the ball and team positioning as two essential factors for strategy analysis. To accomplish this, we estimate the distance between the ball and each player, then we set the ball possession to the relevant team.
Another element that may have an impact on a tactic is the team’s position. This section is motivated by Dynamic analysis of team strategy [10], which involves computing the convex hull, determining the mass’s centre and its distance from the center of the field. The Table 3 shows the calculation for team position terms and the Figure 3 visualizes them.
We outline three hypothetical field segments that characterize team position. One of the following predicates can be generated by the system in relation to the calculation in the Table 3:
7. Rule learner using In this section, we explain how our method learns rules from generated BK. We first discuss the current strategy in each instant of time in section 7.1 and then we propose the prediction rules in section 7.2. We employ by [27] which is the noise-tolerant version of standard Metagol. looks for hypotheses that are consistent with randomly chosen sections of the training instances, then rates each one on the remaining training set to determine which hypothesis received the greatest score.
The BK updates every 200 frames, equivalent to 10 seconds of the video. Thus, the system generates new rules related to the current state. The following rules show an example of one state:
t a c t i c ( A , o f f e n s i v e ) : − d e f e n d e r s ( A , c o n s t a n t ) , a t t a c k e r s ( A , expand ) . t a c t i c ( A , d i f e n s i v e ) : − d e f e n d e r s ( A , expand ) , t a c t i c _ 1 ( A , c o n s t a n t ) . t a c t i c _ 1 ( A , c o n s t a n t ) : − a t t a c k e r s ( A , c o n s t a n t ) , b a l l ( A ,
o u t O f P o s s e s s i o n ) .
The variable denotes the team’s name. The predicate tactic_1 is a predicate invention, generated by Metagol. By creating more predicates, the Metagol algorithm invents more simple rules than a complex one. The hypothesis defensive or ofensive may be accepted for teamA depending on the situation. Our implementation is shown in Algorithm 1. Data from the frames and initial BK are fed into the alongside the parameters and .
← while ← do ← ( ) , ← ( ) ← (, ) ← ∪ ← ( , , , ) end while
In contrast to the current situation, the prediction calls for prior knowledge. As a result, we continue to process the entire set of data sequentially. To determine the most often used strategy, we divide the number of calculated tactics, such as ofensive and defensive by the total number of frames. The outcome and the final state of ball possession are favorable to the BK.
p r e d i c t ( A , o f f e n s i v e ) : − t a c t i c ( A , o f f e n s i v e ) , b a l l ( A , i n P o s s e s s i o n ) . p r e d i c t ( A , o f f e n s i v e ) : − t a c t i c ( A , d i f e n s i v e ) , b a l l ( A , i n P o s s e s s i o n ) . p r e d i c t ( A , d i f e n s i v e ) : − t a c t i c ( A , d i f e n s i v e ) , b a l l ( A ,
o u t O f P o s s e s s i o n ) .
Null Hypothesis 1: train dataset.
Null Hypothesis 2: Hypothesis 1. p r e d i c t ( A , d i f e n s i v e ) : − t a c t i c ( A , o f f e n s i v e ) , b a l l ( A ,
o u t O f P o s s e s s i o n ) .
We evaluate the prediction in the experiment section. The variable A indicates the name of the team, the body predicate given from whole prior frames and ball possession on the most recent frame.
In this section, we evaluate our methodology utilising the MIL technique (Metagol). For comparison with the LSTM model, the evaluation assessed the following hypotheses to determine accuracy:
Null Hypothesis 3: There is no deference between MIL and ILP in terms of accuracy. We examine an LSTM model in the experiments that have three nodes in the output layer that predict values for defenders, midfielders, and attackers. With a batch size of 64, the mean squared error is employed as the loss function. We divide our dataset’s 22961 records into 80% (18300 records) to account for train size.
The train set for Metagol contains 8 examples. The target rule is /2 with body predicates of /2 and /2.
Results in Table 5 compare the predictive accuracy of deep learning model and MIL approach. The test is fairly selected from 18 samples(9 ofensive and 9 defensive), hence the accuracy is set to 50% by default.
The Prolog and Python code and the sample dataset can be found in the following Github repository: https://github.com/danielcyrus/Explanable-Game-Strategy-Rule-Learning-fromVideo.git
This paper studies the learning system using the relational learning and explainability paradigms. We can make use of prior knowledge and visual concept learning tasks by utilising the ILP techniques, including MIL framework.Moreover, Our research demonstrates that MIL methodologies can learn from small sample sizes and outperforms LSTM. Our experiments indicate that our learning system can generate relational rules from background knowledge given from video. This research focuses on tactic analysis in the category of defensive and ofensive of a soccer match. However, a limited number of features can be extracted from a single video (i.e. missing tiny objects while detecting players, captured from a single view and using a distorted video), We produce absolute and precise rules by extending and growing additional dataset [5]. We intend to investigate more intricate predicates and predictions in the future to improve accuracy and to examine more relational and semantic data. the royal statistical society. series c (applied statistics) 28 (1979) 100–108. [17] M. Ankerst, M. M. Breunig, H.-P. Kriegel, J. Sander, Optics: Ordering points to identify the clustering structure, ACM Sigmod record 28 (1999) 49–60. doi:http://dx.doi.org/10. 1145/304182.304187. [18] S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural computation 9 (1997) 1735–1780. [19] S. H. Muggleton, D. Lin, N. Pahlavi, A. Tamaddoni-Nezhad, Meta-interpretive learning: application to grammatical inference, Machine learning 94 (2014) 25–49. [20] A. Cropper, S. Dumančić, R. Evans, S. H. Muggleton, Inductive logic programming at 30, Machine Learning (2022) 1–26. doi:http://dx.doi.org/10.1007/ s10994-021-06089-1. [21] C. Hocquette, S. Muggleton, How much can experimental cost be reduced in active learning of agent strategies?, in: Inductive Logic Programming: 28th International Conference, ILP 2018, Ferrara, Italy, September 2–4, 2018, Proceedings 28, Springer, 2018, pp. 38–53. doi:http://dx.doi.org/10.1007/978-3-319-99960-9_3. [22] A. Cropper, S. H. Muggleton, Metagol system, https://github.com/metagol/metagol, 2016.
URL: https://github.com/metagol/metagol. [23] D. Garrido, D. R. Antequera, R. L. D. Campo, R. Resta, J. M. Buldú, Distance between players during a soccer match: The influence of player position, Frontiers in Psychology 12 (2021) 723414. doi:http://dx.doi.org/10.3389/fpsyg.2021.723414. [24] A. R. Amani, Distance covered and activity analysis of football players during world cup 2014, Archivos de medicina del deporte: revista de la Federación Española de Medicina del Deporte y de la Confederación Iberoamericana de Medicina del Deporte 35 (2018) 376–379. [25] P. Jones, N. James, S. D. Mellalieu, Possession as a performance indicator in soccer., International Journal of Performance Analysis in Sport 4 (2004) 98–102. doi:http://dx. doi.org/10.1080/24748668.2004.11868295. [26] W. Frencken, K. Lemmink, N. Delleman, C. Visscher, Oscillations of centroid position and surface area of soccer teams in small-sided games, European journal of sport science 11 (2011) 215–223. doi:http://dx.doi.org/10.1080/17461391.2010.499967. [27] S. Muggleton, W.-Z. Dai, C. Sammut, A. Tamaddoni-Nezhad, J. Wen, Z.-H. Zhou, Metainterpretive learning from noisy images, Machine Learning 107 (2018) 1097–1118. doi:http: //dx.doi.org/10.1007/s10994-013-5358-3.