According to the World Health Organisation(WHO), cardiovascular diseases (CVDs) are the number 1 cause of death worldwide, claiming nearly 17.9 million lives yearly [1]. Eighty-five percent of all CVD deaths are from heart attacks and strokes. This rising crisis has received relatively small coverage up to date, given its massive economic development effect in countries. However, Early detection of stroke and treatment of it is essential for an excellent outcome. This research aims to discuss the challenge of providing early detection of stroke through drooping mouth detection on Client-Server based architecture, a TeleStroke System (TSS), which is helpful for initial treatments. In this paper, the input data is from the Kaggle and YouTube Facial Palsy (YFP) database. We can find that simple Machine learning gives better accuracy than other deep learning models from the experimental results. It means that we can effectively assist doctors in early detection and diagnosing of stroke.
Stroke is commonly described as a clinical syndrome of presumed vascular origin,
characterized by rapid signs of local or global cerebral dysfunction that usually
lasts for more than 24 hours or even death [
Generally, there are three types of stroke. The first type of stroke is called Ischemic stroke. It occurs when there is a blockage in arteries either by gradual plaque build-up, blood clots, and other fat deposits. The second form is the hemorrhagic stroke that happens when a blood vessel located in the brain splits and induces blood to leak into the brain. This form of stroke is responsible for 13 percent of all strokes, and it accounts for over 30 percent of all stroke-related deaths. Lastly, an Embolic stroke occurs when a clot breaks off from the artery wall and becomes an embolus that can travel further down the bloodstream, resulting in the smaller artery’s blockage. Embolic typically originates from the heart, where uncommon diseases may cause clot formation. 2
Depending on the latest National Stroke Association statistics[
When you deal with stroke, it’s always crucial to know what you want to identify.
It is safer to provide a standardized program that addresses specific common
stroke symptoms. One of the signs of stroke is facial paralysis, meaning, on one
side of the face, there is facial sagging, which is known as face drooping. The
droopy mouth is not only a symptom of stroke but also a sign of a brain and
nerve disorder known as Bell’s palsy. Bell’s palsy, in short, is muscle weakness or
paralysis on one side of the neck. As a result, facial nerve damage that controls
the muscles on one side of the face causes that other side of the face to drop[
A Vision-based interface is provided where the user’s hand, foot, or head
movement are identified and monitored. The framework is designed for
child-tocomputer interaction. They base the device assessment on the concept of HAAT
(Human Activity Assistive Technology) [
It projects the system to be less intrusive and cheaper than other solutions,
in which they built up a video-based prediction process. This function: first, the
movements of different parts of the body are isolated, then the motion
characteristics are extracted and used to identify persons as stable or impaired. The
results show that visually collected motion data allows the Identification of Bell
palsy and Pseudoperipheral palsy as the state-of-the-art detection using data
from electromagnetic sensors [
” Camera Mouse” was introduced in [
They built a visual monitoring system in [
In [
A method was proposed that recognizes three facial expressions in [
They provide a system for recognizing facial emotions in [
In [
They present a system for face recognition and face emotion detection in [
The research focuses on image processing to obtain facial features/landmarks,
mainly mouth corners, determine the droopy mouth threshold value and
deployment using Android Studio. Consequently, the mobile application will address
certain user groups such as neurosurgeons and emergency medical services.
However, the mobile application could also be useful for patients, potential patients
with strokes, and generally any user who is health-conscious anywhere and
anytime[
DATASET
– Stroke Faces on Kaggle[
This segment addresses the proposed TeleStroke system for Eye area and the mouth region. TeleStroke consists of client-server stages: the client-side relates to video pre-processing using face detection and landmark position to locate each sequence frame’s faces. In this phase, the Integrated Deep Model is used. The face images detected are then cropped to the face, and the number of frames per sequence is normalized to a fixed length. The server side consists of various models, such as the ML model, FCNN, CNN, VGG16(Transfer Learning), one for each of the face analysis tasks. The proposed structure for Telestroke is given in figure 1. The front end is a web application comprised of HTML, CSS, JavaScript, AJAX, which ask the user’s permission to grant for camera access that takes live stream image of the user i.e., a WebSocket connection is established. From this stream, a picture is taken out, and the build model is applied. Ajax call is made to get the response and request faster.
This section provides a detailed experimental evaluation of the Stroke recognition system using the TeleStroke Framework. All experiments are conducted on Google Colab with GPU using the Tensorflow 2.0.
The dataset is combined and viewed as one group of droopy as 0 and not droopy faces as 1. This dataset was transferred to different steps and tested on various algorithmic models.
The evaluation methodologies are as follows: 5.1
In this part, the Area ratio of the eye and Slope calculation of mouth plus eye and mouth coordinates are saved in CSV file, and the droopy faces are labeled as 0, and Normal Subjects are marked as 1 which will be used for classification.
The Fig 4. shows the 10-fold cross-validation technique is performed to evaluate each algorithm on training data, which is critically designed with the same random seed to ensure that the equal splits are presented to the training data. Each algorithm is evaluated precisely the same way.
Fig 5 and 6 is the precision, recall, f1 score, and ROC Curve for the Naive Bayes as it was only tested with the testing data. It would indicate from these results of 10 cross-fold validation both Naive Bayes and Random Forest are perhaps worthy of further study on this topic. 6
The first layer has 600 neurons, the second layer has 400 neurons, and the third layer comprises of 200 neurons as shown in Fig 4.7 The lower the loss, the better a model will be (unless the model is overfitted to the training data). The loss is based on training and validation, and its definition for these two sets is how well the model is doing. Contrary to accuracy, the loss is not a number. It is a summation of the errors in training or validation sets that were made for each example.
There are also other subtleties when raising the value of the loss. For example, we may run into the problem of over-fitting, in which the model ”memorizes” the patterns of training and becomes ineffective for the test set. We use Dropout for that reason. Name Epoch Accuracy Loss
FCNN 100 0.95-0.97 0-2 5.3
The amount of ”wiggle” in the loss is proportional to the size of the batch. The wiggle should be reasonably large when the batch size is 1. If the batch size is the maximum data-set, the wiggle should be small, as each gradient update will monotonically increase the loss function (unless the learning rate is set too high).
The difference between the accuracy of the training and evaluation indicates how much overfitting.In Fig 7 and 8, we can observe that too much wiggle is formed, and the model starts to overfit at 0-20 range. The data is not augmented.
The yellow validation curve shows very low validation accuracy relative to the training’s accuracy, suggesting heavy overfitting (note that validation accuracy can also begin to go down after some point). Compared to the above Fig 7 and 8 Fig 9 and 10 is better and the wiggling is less and there is no sign of over fitting. 6
The proposed Web application requires incorporating four critical processes: image acquisition, key points extraction using OpenCV, mathematical measurement, and droopy mouth detection. This paper uses a Simple ML, Fully Connected NN, and CNN architecture with ResNet backbone to present a fully end-to-end framework named Telestroke System for droopy mouth detection. We tested the design using a combination of 3 datasets to get an F1 score of 94 percent, respectively, for the droopy mouth. 7
In this paper, we took on a novel and challenging task of assessing the different models with our proposed model. Although our model performed very well on the selected data sets, there are many areas where it is possible to pursue this research. Firstly, it is possible to achieve hyperparameter tuning and optimization by adding some complex feature vector representations; there is also the potential to investigate more complex backbone and deeper networks, which are limited in this work due to computational overhead 3D CNNs. It opens up a research opportunity for simplifying droopy mouth detection & recognition in Mobile Application.
The current dataset should be expanded to a more consistent one study by including a more substantial number of face photos labeled with demographic details, including age, gender, and ethnicity. As the data collection expands, additional knowledge will become more accurate, and we will consider studying the impact of original populations on paralysis disease and its ranking. Besides, the performance comparison of different pre-trained CNN models can be improved by using more precise facial landmark detection techniques and enabling the recognition of a droopy mouth from tilted or rotated images. Besides, the proposed model could be enhanced to achieve faster identification of droopy mouth and better recognition performance. Will be seen as a potential problem for research. 8
This research was conducted with the financial support of ADAPT Core (Platform & Spokes) under Grant Agreement No. 13/RC/2106 and at the ADAPT SFI Research Centre at Cork Institute Of Technology. Science Foundation Ireland funds the ADAPT SFI Centre for Digital Media Technology through the SFI Research Centres Programme. It is co-funded under the European Regional Development Fund (ERDF) through Grant 13/RC/2106. 10
Rishabh et al.