In the 21st century, IT innovations in sports have become very impactful. Highly innovative equipment allows athletes to stay eficient, reach their best results and even predict the opponent's strategies. Despite the growing interest in sport innovations, adaptive sports for disabled persons still feels the lack of modern solutions. We introduce the solution to analyze the video stream of goalball and using computing power to calculate the total performed throws and help to gather statistical data about the game
analyzer for teams and athletes, “HomeCourt” app [5] for basketball players skills development), game stratIn the 21 century, IT innovations in sports have be- egy prediction and even face recognition. Most of the come very impactful. Highly innovative equipment video tracking approaches are based on points of interallows athletes to stay eficient, reach their best re- est tracking [6], throughout video frames or color desults and even predict the opponent’s strategies. De- tection in three-dimensional color matrix. Video analspite the growing interest in sport innovations, adap- ysis is a complex process and the traditional analysis tive sports for disabled persons still feels the lack of techniques usually requires a lot of calculation and bemodern solutions. comes very costly. Kanade–Lucas–Tomasi (KLT) fea
Contacting the Lithuanian goalball team revealed, ture tracker [7] is one of the approaches to feature exthat nowadays game innovations are a must to have traction that is proposed mainly for the purpose to deal an advantage against opponents, but software solu- with the video analysis special intensity and minimize tions are very expensive or not available. To main- the cost. The method is widely used in face recogtain high results with the minimal cost of hardware, nition or motion tracking, where the diferent frames a system, which tracks the ball in real-time or from share a large number of tracking points. It was tested other video sources and gives the statistical informa- on eight broadcasts and achieved a 94% face track detion about ball positioning and movement is needed. tection rate (the average processing speed was 3.8 fs) To achieve this result, the most important task is to [7]. Meanwhile the KLT algorithm based on points solve a video source distortion from the camera posi- of interest, other popular tracking approach based on tioning problem. a color filtration. A color based video filtration uses the spectrum of three dimensional color matrix and 2. Overview the filtration executed by the desired color code. It requires less calculation and it’s easier to implement, Object recognition from video footage and images is but in some cases it does not provide enough data to widely used technology [1, 2]. Projects like football fully track or recognize an object. The other probgate line and goals detection (such as “Once video ana- lem that the video analysis is facing is a particle delyzer” [3] that helps to explain a game concepts by cre- tection and noise reduction. To retrieve statistical inating animated and professionally designed graphics), formation from video sources requires the most accubasketball ball detection (such as “Hudl” [4] – sports rate video analysis with minimal distortion level. Particle filtering and noise reduction use adaptive target IVUS 2020: Information Society and University Studies, 23 April 2020, model with prediction and correction calculations [8]. KTU Santaka Valley, Kaunas, Lithuania To achieve the most accurate results, the color dis" zygimantas1997@gmail.com (Z. Matusevičius); tribution between diferent gradient positions in the gudauskas.julius@gmail.com (J. Gudauskas) image should be minimal and pixels nearest the cor© 2020 Copyright for this paper by its authors. Use permitted under Creative ners should weight less. The efectiveness of the algoCPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CCoEmUmoRns WLiceonrsekAsthtriobuptioPnr4o.0cIneteerdnaitniognasl ((CCC EBYU4R.0)-.WS.org) rithm depends on the density of pixels in video source, but the high level of intrinsic robustness ensures the will guarantee the same order, regardless of the order, proper performance and accurate filtration. After ana- they were marked. Once we know the coordinates of lyzing the possible image processing algorithms, tools, each game area corner, the calculation can be done for and technical requirements for preferred material, it the bottom edge midpoint of the game area and for the was decided that the C# programming language and bottom edge length - these parameters will be necesEmguCV [9] (a cross-platform .Net wrapper to the Open sary for further calculations. Now we can formulate CV image processing library) contains a needed and equations describing the edges of an untwisted playadaptable functionality to the project. An image pro- ing area. These equations will be used in filtering out cessing library can do plenty of things. EmguCV func- points outside the playing area. tionality provides a prepared video source filtration in diferent color formats and let us filter out objectionable objects. These are the main image-processing = ℎ − (1) operations that provide the majority of needed infor- ℎ pmraotbiolenmt-hsaotlvcionugl.d Ibne tuhsiesdciansef,urththeemrcaainlcuinlaftoiromnsatainond = ℎ ℎ − (2) source needed from the video stream is ball positioning coordinates in each frame.
Using the calculated parameters and the marked area corner coordinates, we can calculate the sin and cos values of the rotation angle for the playing area. Using the calculated trigonometric values, we apply the rotation transformation to the coordinates of the playing area corners.
′ = ′ = − + + (3) (4)
In order to get a real playing field proportion coordinates of the ball positioning and movement in the playing field, it is necessary to use a transformation algorithm. Requirements for the method: > 90%
we adjust the coordinates of the game zone upper cor
To ensure a correct method performance regardless ners: the ordinates of the game zone upper corners of camera position, a playing zone calibration is needed. must coincide and the abscissa must be symmetrical Selected corners of the area will later assist in calculat- to the vertical line passing through the middle point ing the position of the object and determine the suit- of the game zone lower edge. Once we have the exact ability of the coordinates. coordinates of the corners, we can formulate equations
The movement of the object within the calibrated that describe the rotated playing area edges. These area is monitored by examining each frame of the video boundary equations will later be used to perform comaterial and subsequently systematizing the data. ordinate transformations. Equations that used to calThe method pseudocode: culate transformation coeficients are formed as well.
3.1. Game Zone Preparation
3.2. Receiving of Object Coordinates
point of bottom side line
3.4. Movement Vectors Analysis Once the game zone calibration is done, a ball tracking algorithm will begin to track the ball in the video stream. Object tracking is performed using every frame of the video stream and color filtration. In order to recognize objects by color, a frame has to be converted into HSV format. Then, applying the color filter, we can get a binary image where the color of interest is Figure 2: Game zone view after calibration. replaced by white and everything else is black. The binary image may have a lot of "noise" and unnecessary objects. To reduce it, we apply "noise" reduction playing field proportions. This transformation elimoperations [10, 11]. The frame processed in this way inates the asymmetry of the camera angle. Secondly, is already suitable for searching objects in it. the coordinate reference point is moved to the midThe method pseudocode: point of the playing field lower edge. Then a vertical and horizontal scaling factors are calculated and applied (in this case, coeficient equations are used). The scaling factors help to "stretch" the trapezoidal area of playing field from the footage into a rectangle form that is proportional to the real playing field (Fig. 2).
3.3. Coordinates Transformation
nates of the ball in the distorted video frame (Fig. 1). To Once the ball coordinates in the playing area are obget coordinates in proportion to the real playing field, tained, they need to be grouped into vectors. The main we have to perform a sequence of geometric transfor- goal is to add received points into the vectors with the mations. condition: point is added to a vector only when the
First of all, a rotation transformation is performed, distance between the point and the end of the vector is which transfers the coordinates of a point from the appropriate. If a new point is added to the vector, then designated coordinate system of the playing area to the state of the vector is updated: the vector equation the rotated coordinate system that represents a real is corrected and an attempt to complete the vector is executed . Finally, all the completed and eligible vectors moved to the list of completed vectors. If the point cound not be added to the vector, a new vector is created, whose starting coordinates become the point that could not be added to a vector
completed vectors list
For the research and testing of the software solution, The comparison results and calculated accuracy was two diferent video sources were selected, each was displayed in graphs (Fig. 5, Fig. 6, Fig. 7, Fig. 8). rendered into diferent frames per second ratio – 15 Other important factor of the algorithm accuracy fps, 30 fps and 60 fps. is the time duration that it takes to calculate throws
The main goal of the analysis is to investigate the from video stream. Analyzing the first video from Rio accuracy of the algorithm, regarding the requirements Olympics 2016, male match Algire vs. Germany, the mentioned in third section. First selected video for the time duration that it took to analyze the entire video research was male match between Algeria and Ger- in 15 fps (original video length 20:36), was 17:08 (video many teams in Rio Olympics, 2016. To get the accu- stream had a 120% of the original speed), meanwhile in racy of the algorithm, we calculated the total throws the 30 fps video it took 26:40 (video stream had a 77% of performed by each team by hand and compared the the original speed). The analysis in 60 fps took 1:00:24 results with the output after software processing. The (video stream had a 49% of original video speed). comparison results displayed in chart (Fig. 3). Analyzing the second video from Rio Olympics 2016,
From the following data, we can calculate the accu- female match USA vs. Japan, the time duration that racy of recognized vectors and get the average accu- it took to analyze the entire video in 15 fps (original racy in this match. Results was calculated in diferent video length 46:18), was 31:55 (video stream had a 145% frame ratio and displayed in chart (Fig. 4). A second of the original speed), meanwhile in the 30 fps video selected video for the research was female match be- it took 58:24 (video stream had a 79% of the original tween USA and Japanese teams in Rio Olympics, 2016. speed). The analysis in 60 fps took 1:56:39 (video stream Using the same calculation principle, the number of had a 40% of original video speed). throws was calculated by hand and compared with the result of the software analysis in diferent frame ratio.