This research explores technology integration in sports, focusing on video-assisted performance diagnostics for swimming. Motion capture techniques, particularly using machine learning models, aim to automate stroke classification, ofering eficient analysis of swimmers' techniques. The study proposes a hybrid approach with VGG16 for feature extraction and a Random Forest classifier for stroke classification. Despite challenges stemming from limited data, resulting in an accuracy of 0.28125, the study emphasises the potential of deep learning neural networks for both feature extraction and classification with the aid of larger datasets in the context of swimming performance analysis.
In recent times, technology has become a crucial part of the world of sports. From being used to scrutinise and analyse athletes’ performance during their training period to identifying minor edges that can lead an athlete to victory, cameras and performance monitoring systems are used at every stage of the journey. Applying motion capture techniques through video cameras can go a long way in enhancing a person’s capabilities and preventing risks of injuries and player fatigue.
In today’s day and age, video-assisted performance diagnostics in the context of swimming have become an indispensable part of improving and enhancing a swimmer’s technique. This sort of evaluation of stroke rates, the body postures, and the diferent phases of a stroke cycle are of great importance to athletes to help them understand how to make movements with minimum movement economy and maximum speed. But it is still largely done manually by replaying and physically noting important features from the video playback. This is not only labour-intensive but also exhausting and time-consuming. Automating this process using state-of-the-art models will largely help in catering to swimmers and athletes who do not have the skill and knowledge to evaluate their technique and are largely dependent on skilled experts for the same.
Using machine learning models to detect the type of swimming stroke is an eficient way
to analyse sports performance and derive conclusions about the improvements required in
terms of technique and posture. The task [
In a study by Hosseini Fani [
In another study on recognising basketball turning and dribbling, Zhang et al. [
Furthermore, a study [
A study [
Another study [
The dataset comprised 96 labeled images which were partitioned into an 80 per cent training set and a 20 per cent validation set. The images belonged to 4 classes - backstroke, freestyle, breaststroke, and butterfly, as shown in figures 1-4, with the class-wise distribution being 24.
Convolutional Neural Network is a widely used machine learning model and is a fundamental element of deep learning algorithms.
A CNN consists of four components: A convolutional layer, an Activation operation, a
Pooling layer and a Fully Connected layer. The convolutional layer is the heart of the neural
network and is responsible for feature extraction. It involves applying a filter to the image to
identify and locate significant features. An element-wise multiplication is performed between
the two-dimensional array of weights called the filter and the two-dimensional input array and
the result is summed up. This computation is performed using a sliding window [
The second component of the CNN is the activation operation. This component is responsible for recognizing specific features in the images it is being trained on and allows the network to learn the non-linear relationships between the weighted sum of the inputs and the output. The most widely used activation functions are Rectified linear unit (ReLU), Sigmoid and Hyperbolic tangent functions.
The third part comprises the pooling layer that is used to reduce the size of the feature map which leads to dimensionality reduction. This sort of down sampling helps in maintaining a lower-resolution feature map while retaining the important features that are important to the classification task. There are largely two types of pooling: Average pooling and Maximum Pooling.
The final component is the Fully Connected(FC) layer. The FC is a densely connected layer whose weights and biases are learned during the training process. This layer’s functionality is to flatten the 2D feature map obtained from the previous layers into a 1D array and output a score for each one of the classification classes. 4.2. VGG16 VGG 16 represents a convolutional neural network designed for image classification tasks. Pre-trained on the extensive ImageNet database, this model serves as a feature extractor, demonstrating its proficiency in learning hierarchical representations of visual features. Comprising a total of 16 layers, VGG 16 consists of 13 convolutional layers followed by 3 fully connected layers. The model operates by constructing a 3D tensor, which is subsequently flattened into a 2D vector before being fed into the Random Forest Classifier. The utilisation of a pre-trained model facilitates the extraction of intricate and abstract features. 4.2.1. Random Forest The Random Forest algorithm serves as an ensemble classification method that leverages the collective decision-making capabilities of multiple decision trees. This method involves the aggregation of individual decision trees to collectively determine the class output. The fundamental unit of the Random Forest is the decision tree, and each tree is trained independently on a randomly selected subset of the training data. An advanced form of the bagging algorithm is employed, introducing an element of randomness to enhance the diversity of the individual trees within the ensemble.
The network architecture consists of subsequent Conv2D and MaxPooling2D layers, with the Conv2D layers employing 32 filters of size 3x3 to learn the hierarchical representations. This layer is followed by a MaxPooling2D layer with a 2x2 pooling size, providing a degree of translational invariance. The third Conv2D layer introduces 64 filters, succeeded by a MaxPooling2D layer with a 2x2 pooling size, resulting in a down-sampled output that is reshaped into a one-dimensional array. The architecture includes two dense layers, helping the model understand complex combinations of lower-level features related to diferent strokes. To enhance regularisation, a Dropout layer with a dropout rate of 0.24 is applied. The architecture employs Rectified Linear Unit (ReLU) activations after each convolutional layer ensuring nonlinearity and spatial down-sampling. Additionally, Softmax activation in the final layer facilitates the generation of output class scores through probabilistic distributions. 5.1.1. VGG16
The implementation leverages the VGG16 architecture, consisting of a total of 19 layers with 16 convolutional and 3 fully connected layers. However, only the 16 layers comprising the convolutional base of VGG16 are utilised for the specific purpose of feature extraction. The convolutional layers employ 3x3 filters with a stride of 1, facilitating precise pixel movement. The number of filters increases with the depth of the model, and the architecture is characterised by multiple Conv2D and MaxPooling blocks, each contributing to the extraction of intricate features. Max pooling is executed with 2x2 windows and a stride of 2, eficiently down-sampling the input. Rectified Linear Unit (ReLU) functions are applied to activate the convolutional blocks, providing non-linearity to the model. Addressing the advantages of using smaller filter sizes, hierarchical feature learning and parameter sharing contributes extensively to capturing ifne-grained details and reducing the risk of overfitting. The dataset was used to further train VGG16, which was pre-trained incipiently on the extensive ImageNet dataset. Figures 5 and 6 represent the feature maps extracted in each filter before and after the application VGG16 layer.
The final accuracy of the random forest model on the test data was 0.28125. This low accuracy score is a result of training a neural network like VGG16 on a smaller dataset consisting of 96 images. To accommodate the limited nature of the dataset, we performed data augmentation and opted to use hybrid models that combined deep learning neural networks like VGG16 for feature extraction and traditional models like Random Forest for classification. The CNN model was also applied on the training data which yielded a low validation accuracy of 0.30 due to the extensive dataset requirement of the model. We aimed to extract the most significant features by leveraging state-of-the-art technology and tuning it to fit our dataset by making changes in the hyperparameters, but it underperformed and overfit due to the smaller size of the training dataset.
Through the scope of this research, we implemented a hybrid approach. The capability of the CNN and VGG16 models to extract and capture fine-grained details was utilised in identifying the significant features in the images consisting of swimming strokes to correctly classify them.