In elite soccer, decisions are often based on recent results and emotions. In this paper, we propose a method to determine the expected winner of a match in elite soccer. The expected result of a soccer match is determined by estimating the probability of scoring for the individual goal scoring opportunities. The outcome of a match is then obtained by integrating these probabilities. In our experimental study, we show that the probabilities of goal scoring opportunities accurately match reality.
The use of advanced big data analytic in soccer starts showing its potential,
however, sports analytics as a research area is still only emerging. Some of the
problems are well-de ned, e.g. many studies have attempted to predict the
result of soccer matches before the match actually started. Various perspectives
have been used to tackle this problem. A common perspective to look at this
problem is the prediction of soccer matches from a betting perspective [
Other problem formulations are less straightforward to formalize, e.g. providing insights into how well each of the teams or individual players did in the match. Here the approaches range from plotting histograms to heatmaps aligned with the playing eld, and from counting successful actions to computing complex features based on domain knowledge.
In our work, we take a complementary perspective. We consider the use of predictive modeling to explain the outcome of a match based on the available data from the match (rather than trying to predict the outcome of the game before the game starts).
By explaining the match outcome we mean accumulating evidence of which team should have won the match based on the created goal scoring opportunities and accounting for both the quantity and quality of such opportunities. The demand for such an approach comes from soccer clubs themselves. These soccer clubs often base their decisions on recent results, even if they do not completely understand where these results come from.
In this paper, we provide an empirical illustration that inducing a predictor from the past soccer matches and applying it on the current match data provides us with accurate probabilistic estimates of scoring opportunities to result in goals. 2
An important aspect of this paper is to lay the sound foundation for reasoning about scoring opportunities. We shall be able to get insights into two kinds of questions: \how can we quantify the value of a shot given the scoring opportunity?" and \how can we quantify the value of a goal scoring opportunity created by a team (disregarding whether it was realized or not)?" If we can provide good estimates, then someone can see how many opportunities (and their quality) each of the teams produced during a match and how many of them each team realized.
In particular, if we get probability estimates for each scoring opportunity, we can simply sum up this estimates and get an expected number of goals as follows from the Poisson binomial distribution. Thus, if we denote pi be the probability that we scored a goal in the scoring opportunity i, and model each i as a Bernoulli random variable yi Ber(pi) then the expected number of goals in the match of n n n n scoring opportunities is equal to: E[#goals] = E( P yi) = P E(yi) = P pi. i=1 i=1 i=1
The idea of applying predictive modeling for quantifying the quality of scoring
opportunities is not new. E.g. logistic regression was used in [
Formally, we learned a classi er that is functional mapping y = f (X), where for each scoring opportunity Xi that is a feature vector describing it and any additional contextual information, the classi er should accurately predict yi 2 fgoal; no goalg. The classi er must generalize well to previously unseen scoring opportunities and avoid over tting to the training data. We expect that a classi cation technique that is not only accurate (i.e. has low error, bias, and variance) but also can provide good con dence estimates for the predicted output would work best for our purpose.
One could argue that training the classi ers on only the best players of the world would lead to a more accurate and insightful model of desirable player performance. Such a model would, however, have only limited business value since it would not be directly applicable to poorer players. Adding the player quality to the model allows the model to learn the relations between player quality and the value of a goal scoring opportunity. The scores could be corrected accordingly. Furthermore, the quality of a goal scoring opportunity is in uenced by the opposing team. Since the location of the defenders is already de ned, the in uence of the defenders is likely to be limited. The goalkeeper, however, could have signi cant in uence on the quality of a goal scoring opportunity.
Consider two situations in which a player attempts to score with the only di erence being the opposing goalkeeper. If one of these goalkeepers would be the best of the league and the other goalkeeper would be an average goalkeeper. Intuitively, these situations would not have the same probability of resulting in a goal. Therefore, the quality of the opposing goalkeeper taking into account.
Since goals are so rare in soccer, more non-goals than goals exist in the data
set. Therefore, a combination of over-sampling (SMOTE) [
We also apply calibration to make the scores obtained with the classi er to
be interpreted as a probability of scoring a goal given the scoring opportunity
situation. The classi cation algorithms provide class membership probabilities,
i.e. the con dence a sample belongs to a certain class. These class membership
probabilities can not be interpreted as the probability that a goal attempt results
in a goal. Calibrating the classi er ensures that its output can be interpreted as
a probability that a goal attempt results in a goal. Two main calibration
techniques exist: Platt's scaling [
With the use of classi cation algorithms and calibration techniques, point estimates can be determined for the goal scoring opportunities. In order to avoid misleading interpretation of the quality scores, we also estimate prediction intervals. To determine the standard deviation similar goal scoring opportunities, the samples are rstly clustered. The standard deviation of the samples in the cluster is then used to determine the prediction intervals. Gaussian Mixture clustering is used since this technique is often used in kernel density estimation. Intuitively, if the variation of the point estimate is too large, no valid statements about individual point estimates can be made. However, aggregating the data, however, still valid statements can be made due to the law of the large numbers. 3 3.1
We had access to three di erent data sources are available: 1) data about the
main events during a match tracked by (employees of) ORTEC; 2) data about
the quality of players [
In total, we have data from seven di erent leagues over three seasons. This leads to a total of 5017 matches in which 128667 goal attempts were performed. Of these goal attempts, only 14109 resulted in a goal.
It is worth noting that each data source has its own data quality problems that can a ect classi ers and the conclusions derived from their outputs. The data quality issues of the ORTEC data are related to the tracking of the events by ORTEC employees who have to select the location of the event at the right time and at the right location that is hard to do, especially in a near realtime setting. The main data quality issue with the FIFA data comes from the determination of the stats that is somewhat vague and could be incorrect for some of the players. Finally, the data quality issues of the Inmotio data come from the cases in which the cameras lose the correct player or accidentally selects the wrong player. In this case, the location of the player is incorrect and it is di cult get the correct location. We had too little data from the cameras and hence did not use it for inducing classi ers.
With the use of the considered data sources, various features can be extracted. A list of the extracted features for each data source is provided in Table 1.
Context
Part of body
Dist to goal
Angle to goal
Originates from
We experimented with four di erent classi cation techniques algorithms (as
implemented in scikit-learn [
Figure 1 illustrates two examples of probabilities of the individual goal scoring opportunities obtained with our approach.
The features for these examples and the probabilities are shown in Table 2.
Since we want our classi ers to provide higher scores for better goal scoring opportunities, we report AUC performance, but also provide the precision, recall, (a) Example 1 (b) Example 2 (c) Example 3 and F-score value for reference. Table 3 summarizes the results. Also the standard deviation of the AUC over the di erent cross validation phases is provided. We can see from the table that Random Forest performs reasonably well and outperforms other classi ers.
Next, we perform the calibration step to make the scores more accurate. We
used the reliability graph introduced in [
Figure 2 shows that the predicted values are indeed close to the actual ratios of goal scoring opportunities resulting in goals. We also show the con dence intervals of the predicted values in the bins. Since these con dence intervals
Calibration plots (Brier score 0.0815) 1.0 0.8 iftssooonp ive0.6 irF tca0.4 0.2 are narrow, it is safe to consider the scores as the probability of a goal scoring opportunity to result in a goal.
We also report the Brier score [
To determine whether the expected goals can be used to explain match results, the predicted match outcomes are compared to the actual match outcomes. Table 4 provides the number of matches that are correctly predicted with the use of the expected goals model. Furthermore, the number of games where the number of predicted goals was at most one goal o is shown, the number of matches in which the result was correct (win team 1, draw, win team 2). Finally, the Mean Squared Error (MSE) for the number of goals per match is provided.
What stands out from Table 4 is that in only 1366 of the 5020 matches, the exact score of the match was predicted based on the expected goals. If, however, one goal di erence is accepted, 3443 of the 5020 matches have correctly predicted scores. Therefore, it seems that the expected goals model is, in most cases, almost correct. The MSE Match strengthens this statement. The MSE match shows that the average MSE of the result of a match is 2:366. Therefore, the average number of goals predicted di erence goals of both teams di ers p2:366 1:538 from the actual di erence in goals.
So far, just the exact results are examined. Maybe even more interesting, is how often the expected goals model predicted the correct winner. This is given by the number of correct results in Table 4. Obviously, the number of correctly predicted matches is higher than the correctly predicted scores. What stands out, however, that the number of correctly predicted matches is not close to the number of scores predicted correctly where one goal di erence was allowed. This shows that games where the model is one goal o in the match, this one goal also in uences the result of the match. To evaluate in which cases the one goal di erence most often in uences the result, the problem of predicting the winner of a match is de ned as a three class problem where either Team 1 wins, Team 2 wins or the game ends in a draw. The confusion matrix of the tree-class problem is provided in Table 5.
We consider three immediate applications of the quality scores by soccer clubs: 1) performance evaluation over a given period of time; 2) analysis of matches and 8
team performance; and 3) assessment of players and individual training sessions management. Soccer clubs tend to base decisions on results of a short period of times and emotions. Since many factors in uence the results of soccer matches, these results could not closely match the reality. These decisions could, therefore, be based on misperceptions. A more objective metric of the quality of goal scoring opportunities would provide a more objective decision-making strategy. Before ring sta , for example, an expected league table could be created to determine whether the team is actually performing badly.
Furthermore, the results of matches could be plotted together with injuries, suspensions, red coaches and many more factors to nd out the relation of the events that happened during a season on the performance of the team. Goals are very rare in soccer that leads to high in uence of a single goal on the result of a match. By analyzing goal scoring opportunities instead of actual goals scored, a more objective way of analyzing the result is obtained. Adding the quality of the goal scoring opportunities makes this analysis even better.
Figure 3 provides an illustrative example of simple visualization of match scoring opportunities and whether they were realized. On the left side of this gure, the progress of a match in terms of expected goals is provided. Here, one could see that the upper team (represented by red), should have been in front from the beginning of the match and had the upper hand during the match. The right side of the gure shows from which players the expected goals (the bars) and the actual goals (the numbers) were coming. Since this data is classi ed, the names of the players on the x-axis are removed.
Furthermore, similar to the analysis for periods of time, the expected result of a match could be plotted over time. By adding important events such as goals scored, cards, substitutions and many more factors, the in uence of these factors could be researched in more detail. 5.3
A major advantage of determining the quality of the individual goal scoring opportunities is that it generates more possibilities than only determining match results. Aggregating over players, instead of matches, leads to insights into player performance. These insights could be used to evaluate players, adjust training programs or perform player acquisition.
An example of interesting insights from the expected goals is when the expected goals are plotted for di erent locations on the eld. This could, for example, show that a player is often shooting from one speci c part of the eld but never scores. If the probabilities of these goal scoring opportunities are high, the player is obviously doing something wrong in these cases and his actions could be analyzed in more detail. If, however, the probabilities are low for a player on the eld, but that player shoots very often, someone could point out to him that shooting might not be the best decision at that part of the eld.
Another example comes from the case where players, especially strikers, score many goals in one season (take for example Jamie Vardy of Leicester City during the English Barclays Premier League in 2015-2016). Those strikers are often bought by big clubs since they did score a lot. It could, however, be the case that such a player did score a lot but had a much lower amount of expected goals. This could suggest that the speci c player was lucky during that season. Of course, more research has to be performed on that player's performance, but the expected goals indicator could be a useful tool in player acquisition. 6
In this paper, we presented a method with which the results of soccer matches can be described in a more objective manner by evaluating the quality of the goal scoring opportunities for both teams during that match. It is shown that the proposed method performs well in terms of classi cation performance as well as on calibration of probabilistic estimates. Further applications of the expected goals are given by evaluating seasons, matches, and individual players.
An important point to make when using the probability estimates of the goal scoring opportunities is that these estimates may have a high standard deviation. The scores for goal scoring opportunities, could, therefore vary quite a bit, even thought the goal scoring opportunities are similar. Users should, therefore, be very careful when making statements of individual goal scoring opportunities with too few point estimates.