Frail is a sub-health status, which indicates the weak and delicate status of a person. Also, frail is a common problem in modern society. For frail diagnosis, people should go to a clinic which is not convenient and especially hard for the elderly. This paper aims at proving the possibility of using a Doppler radar image for frail status diagnosis of the elderly. In this research, we ask for the help of 178 elderly for creating a dataset by questionnaire and taking walking action by Doppler radar. The reason for selecting the walking action is it is the most normal activity in daily life, and a lot of health information is hidden in this action. And radar image is a good method of privacy protection than the RGB camera. Seven machine learning methods, including the proposed Neural Network, are used for the frailty classification by using the walking Doppler radar images of the elderly. Before making a classification, image pre-processing is proposed for feature extraction. The pre-processing separates every Doppler radar image into four parts in the vertical axis and the horizontal axis, respectively. Then feature point numbers are counted in every separated part after binarization for creating the eight features. In binarization, optimized thresholds are obtained by sliding the threshold in experimentation. The experimental results show that, when utilizing the green channel, the proposed Neural Network achieved 75% prediction accuracy in frailty classification, which outperforms the other machine learning methods. Although the accuracy is not so high as the first research in this area, the possibility of frailty classification using Doppler radar images for elderly persons is proved. The prediction accuracy shall be increased as our future work. * Masterminded EasyChair and created the first stable version of this document † Created the first draft of this document
Frail is a disease, including diminished motivation not attributable to a diminished level of consciousness, cognitive impairment, or emotional distress(Marin, 1990; RS, 1991; RS1 et al., 1991), which has a deep relationship with other diseases such as Parkinson’s disease, Alzheimer’s disease, stroke, are often happen in the elderly people and threaten their life (J et al., 2015; AM et al., 2001; JL et al., 2005; Caeiro et al., 2013). They also noticed that Alzheimer’s disease was found in frailty about half of 47% (JL et al., 29 2005). However, for frailty diagnosis, the elderly should go to a clinic that is not convenient and hard for them elderly. Currently, a few researchers have developed a computer version system to assistant the frailty diagnosis in remote operation. However, the system uses face images which have a problem in privacy protection. And the patients should have subjective symptoms for asking the doctor. Furthermore, frailty usually does not have subjective symptoms, especially be founded by the solitary elderly. Hence the elderly may delay the diagnosis and lost the best treatment period.
As we have known, the world is facing the elderly society which a very high speed, especially in Japan. In Japan, the percentage of the population aged 65 and over (percentage of the elderly) is 28.1%. The population aged between 65 and 74 years is 13.9% and aged 75 years and over is 14.2% in 2018. By 2065, more than 65 years will arrive at 38.4%, more than 75 years will arrive at 25.6% (Japan, 2019). Hence, convenient frailty diagnosis has become much more significant than before. The convenient frailty diagnosis helps the elderly do the early examination and reduce the risk of health problems. This paper tries to use Doppler radar image(DRI) for the frailty classification of the elderly which will help the early access of frailty. Doppler radar images are not using face images which may protect privacy very well. And the Doppler radar may equip in the room and checking the frailty every day without any previous preparation. Unfortunately, little work is done in the area, causing no research to show which action is fit for frailty classification. Hence, We select the most normal activity in daily life – walking. This is mainly because walking has a deep relationship with a health condition. In early1984, walking abilities are used for clinical gait assessment in the neurologically impaired (MK et al., 1984). It is easy to understand the walking action has a relationship with a health condition, for example of stroke patients often difficult for them to control their body when walking. Late, researchers found that the working action linked lots of people’s health information including age (Makihara et al., 2011; Handri et al., 2009), chronic illness(Pitta et al., 2005; Rabinovich 51 et al., 2013; John et al., 2009). In terms of dataset creation, we ask the help of elderly people (more than 65 years old) for creating the Elderly Person Frailty Doppler Radar Image dataset (EPFDRI-dataset). Every elderly people makes a questionnaire for obtaining the frailty or non-frailty, and do a walking action under the Doppler radar and obtaining the experimental images. This paper aims to combines image processing and machine learning method for frailty classification by using the EPFDRI-dataset.
A simple image processing is proposed for extracting the features from the radar images as preprocessing. In simple image processing, a walking Doppler radar image is separated into four parts by the vertical axis and horizontal axis, respectively. Binarization is applied for counting the feature of eight parts for training and classification by machine learning. We apply three patterns for binarization, which includes only using the red channel, green channel, and blue channel. And the threshold of binarization is a slide from 100 to 220 for obtaining the optimized value. Finally, the eight feature point numbers are used for the frailty classification.
In classification, this paper equips seven machine learning models, which include Support Vector Machine(SVM) (V., 1998), K-nearest neighbor (KNN)(S., 1992), Naive Bayes, Decision Tree(J.R., 1886), Random Forest(L., 2001), Ensemble Model(D. and R., 1999) and a proposed Neural Net work model for the frailty classification. This is the first paper that challenges the frailty classification by using Doppler radar images of walking action for the elderly, we want to prove the possibility and effectiveness of this frailty classification by privacy protection and convenient method. The contribution of this work is shown in the following: The Elderly Person Frailty Doppler Radar Image dataset (EPFDRI-dataset) is the first dataset for frailty classification of the elderly in walking action. This research challenges to use of the Doppler radar image of walking action for frailty classification, which is the first privacy protection and convenient frailty classification method.
Image processing and machine learning are firstly proposed for the frailty classification of the elderly. The optimization is discussed, including the threshold of binarization, color channel, and machine learning models. Section 1 discusses related work including frailty classification, health care research of walking, and Doppler radar image research in health care. Section 2 shows the dataset creation. Section 3 introduces the proposal and machine learning method which is used in the experimentation. Also, the experimentation results of the frailty classification are shown in section 4. Section 5 discusses the contribution and the limitation of this research and gives a summary and mention of future work. 2 Creation of doppler radar frailty image dataset for frailty diagnosis
For training the machine learning model and testing the accuracy of frailty classification, the Elderly Person Frailty Doppler Radar Image dataset (EPFDRI-dataset) is created in this research. 178 elderly people help us to create the dataset of DRI as participants. They also cooperate with the questionnaire for frailty classification previously. About the details of elderly participants, 81 people are between 65 to 75 years old, and 98 are between 76 to 94 years old. All of the elderly are Japanese. Besides, the questionnaire and the answer are written in Japanese.
Figure 1(a) is an example of a Doppler radar image with walking action, and Figure 1(b) shows the walking process which is taken. Figure 1(c) shows the experimental environment, the radar size is about 53cm, and the height is 62cm, the start point is about 70cm from radar. The walking distance is 100cm.
This section proposes a frailty classification method by combining image processing and the machine learning method. In detail, image processing is proposed for extracting features from the walking radar image. And then, seven machine learning methods include the proposed Neural Network method are applied for classification by using the extracted features. This paper tries to prove the possibility of frailty classification by machine learning for the elderly. It also searches for the best method and the optimal parameters by experimentation. 3.1
For the reason of the first challenge, which channel in the image is important for the frailty classification is not proved. We focus on pixel configuration for the binarization. Here, we apply three kinds of binarization consists only using the red channel, green channel, and blue channel. The threshold is a very important parameter in binarization, which may influence classification accuracy. The following function shows the threshold decision, in the case of a pixel is greater than the threshold its value is set into 255, otherwise, the pixel value is set into 0. In this paper, the pixel is set as three kinds, including only using the red channel, green channel, blue channel. We slide the threshold from 50 to 220 for selecting the best threshold.
,
{
255 ( ,
0 (
≥ ℎℎ)
ℎ)
(
After the binarization, every image is separated into four parts by the vertical axis and horizontal axis respectively. The white pixel number of eight parts in the binarized image is counted. Finally, the eight numbers are decided as a feature vector for frailty classification by the following machine learning methods. 3.2
K-nearest neighbor is the most simple traditional classifier, by calculating k closest training in the feature space (S., 1992). Absolute distance measuring, Euclidean distance measuring, or some other distance function are used for calculating the minimum distance. In this research, for classification the frailty and non frailty, K is defined as two. 3.3
Naive Bayes is a simple technique for constructing classifiers, Abstractly, Naive Bayes is a conditional probability model: given a problem instance to be classified, represented by a vector representing n features, it assigns to this instance probabilities p, for each of K possible outcomes or classes C. The problem with the above formulation is that if the number of features n is large or if a feature can take on a large number of values, then basing such a model on probability tables is infeasible. Therefore, we reformulate the model to make it more tractable.
A decision tree consists of a decision diagram and possible results (including resource costs and risks), ready to create a plan for the goal. The decision tree is a special tree structure to establish and reserve decisions. The decision tree is a decision support tool that uses a tree-like graph or decision model, including random event results, resource elderly, and future use. It is a solution. Decision tree in operations research, especially in decision analysis, helps to determine a strategy that can most likely achieve the goal. If in practice, decision-making can be adopted online without complete knowledge, a decision tree should be used as the best model for the model or an online selection algorithm model. Another use of trees is as a descriptive means of calculating condition statements.
Random forests is a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest (L., 2001). The point is to create a group of decision trees with low correlation by using randomly sampled training data and randomly selected explanatory variables. First of all, m training sets are generated by the bootstrap method. Then, for each training set, a decision tree is constructed. 4 Experiment and evaluation works
178 Doppler radar images of 187 elderly people are used for the experimentation. In detail, 142 of them which consists of 62 frailty and 80 non-frailty are used for training, and the left 36 images which consist of 16 frailty and 20 non-frailty are used for testing. It has to be mentioned that each person only has one Doppler radar image of walking action. The programming language of Python3.7 is used for feature exaction and machine learning design. Anaconda is used as the standard platform. The hardware environment is CPU: Core i7 8th Gen, Memory: 32GB. VGG16, VGG19 (K. and A., 2015), ResNet152V2 (He et al., 2015), Inception (C. et al., 2016), InceptionResNetV2 (Szegedy et al., 2016), Xception (Chollet, 2017), and MobileNet (Howard et al., 2017). However, few models converge well, shows that the current deep learning method failed to tackle the frail classification by using walking Doppler radar images.
The experimental results try to prove the effectiveness of frailty classification by using radar images in the walking action with the machine learning method. In terms of the feature extraction, binarization is applied by only using the red, green, blue channels, and the threshold is a slide from 100 to 220. The experimental shows that the method can not bring a good accuracy in only using the blue channel. We discussed five machine learning methods for the frailty classification. In conclusion, in the green channel, the threshold slides from 160 to 200 achieve more than 75% of accuracy. It proved that the effectiveness and signification of this research. Even if the accuracy should be improved in the future, this research gives a good signification for proving the possibility of realizing privacy protection and convenient Frail classification method for the elderly person.
The limitation of this research is that the dataset is of small scale, only 178 elderly people are participants who help in creating the dataset. Even if it is very hard to establish the current dataset, and some current research only uses Dozens of participants such as paper (Happy et al., 2019) has 45 participants, and paper (Liu et al., 2018) has 30 patients. For improving the accuracy and realizing the practical implementation, the dataset should be enlarged. Besides, the setting place of Doppler radar and the walking action is the limitation of the research. For the reason that this is our first challenge, the Doppler radar is set in front of the participants, and the walking action is done by order. As means for realizing the production and use it in daily diagnosis, these limitations should be considered.
This paper tries to use the walking action DRI for the classification of frailty for elderly persons. The walking action is the most normal activity in daily life, and radar image is a good method for privacy protection than the RGB camera, using walking action DRI may help us to realize a convenient and privacy-protected method. For the frailty classification, an image processing and machine learning method combined method is proposed. 178 elderly people help to create a dataset by questionnaire for obtaining the frailty or non-frailty, for training and testing machine learning models. Image processing consists of binarization, image separation, and feature pixel counting as the action of feature extraction. We focus on pixel configuration for the binarization and slid the threshold from 100 to 220 for obtaining the optimized value. Seven machine learning methods include the proposed NN method is applied for classification by using the extracted features. In conclusion, in the green channel, the threshold from 160 to 200 achieve more than 75% of accuracy. It proved that the effectiveness and give a good signification for proving the possibility of realizing a privacy-protected and convenient frailty classification method for the elderly person. However, the capacity of the dataset limited the accuracy of our proposal, hence increasing the dataset and improving the accuracy are major works in the future.
[3] Michael Calonder, Vincent Lepetit, Christoph Strecha, and Pascal Fua (2010) “BRIEF: Binary Robust Independent Elementary Features”, European Conference on Computer Vision [4] H. Bay, and T. Tuytelaars (2006) “Surf: Speeded up robust features” Computer Vision ECCV [5] E. Rublee, and V. Rabaud (2011) “Orb: an efficient alternative to sift or surf”, IEEE International Conference on Computer Vision
[6] M. Asada, T. Tanaka, and K. Hosoda (2000) “Adaptive Binocular Visual Servoing for Independently Moving Target Tracking”, IEEE International Conference on Robotics and Automation [7] D. K. Patel, and P. A. Bachani. (2013) “Distance measurement system using binocular stereo vision approach” International Journal of Engineering Research and Technology 2 (12): 2461-2464.
[8] C. Pham and J. Jeon. (2012) “Domain transformation-based efficient cost aggregation for local stereo matching” IEEE Transactions on Circuits and Systems for Video Technology 23 (7): 1119-1130, 2012.
[9] Y. Yoshida (2014) “Gaze-Controlled Stereo Vision to Measure Position and Track a Moving Object: Machine Vision for Crane Control”, Sensing Technology: Current Status and Future Trends II, Springer, Cham
[10] Zelin Meng, Xiangbo Kong, Lin Meng, and Hiroyuki Tomiyama (2018) “Coordinate Transformations for Binocular Vision Systems”, International Symposium on Advanced Technologies and Applications in the Internet of Things, IEEE
[11] Zelin Meng, Xiangbo Kong, Lin Meng, and Hiroyuki Tomiyama (2018) “Distance Measurement and Camera Calibration Based on Binocular Vision Technology”, International Conference on Advanced Mechatronic Systems, IEEE
[12] Zelin Meng, Xiangbo Kong, Lin Meng, and Hiroyuki Tomiyama (2019) “Stereo Vision-Based Depth Estimation”, International Conference on Artificial Intelligence and Data Engineering, Springer [13] Zelin Meng, Xiangbo Kong, Lin Meng, and Hiroyuki Tomiyama (2019) “Lucas-Kanade Optical Flow-Based Camera Motion Estimation Approach”, International SoC Design Conference, IEEE
[14] Zelin Meng, Xiangbo Kong, Lin Meng, and Hiroyuki Tomiyama (2019) “Camera Motion Estimation and Optimization Approach”, International Conference on Advanced Mechatronic Systems, IEEE
[15] C. Stentoumis. (2014) “On accurate dense stereo-matching using a local adaptive multi-cost approach” ISPRS Journal of Photogrammetry and Remote Sensing 91: 29-49.
[16] L. Basaca-Preciado. (2014) “Optical 3D laser measurement system for navigation of autonomous mobile robot” Optics and Lasers in Engineering 54: 159-169.
[17] R. Correal and P. Gonzalo. (2014) “Automatic expert system for 3D terrain reconstruction based on stereo vision and histogram matching" Expert Systems with Applications 41 (4): 2043-2051.
[18] Ogun and S. Philips. (2014) “An active three-dimensional vision system for automated detection and measurement of surface defects” Journal of Engineering Manufacture 228 (12): 15431549.
[19]Zhiyu Zhang, Zhichen Wang, Hiroyuki Tomiyama, and Lin Meng, "Deep Learning and Lexical Analysis Combined Rubbing Character Recognition," The 2019 International Conference on Advanced Mechatronic Systems (ICAMechS 2019), 8.2019. (Ritsumeikan University).
[20]Zhiyu Zhang, Yuxi Chen, Hiroyuki Tomiyama and Lin Meng, "Rubbing Characters Recognition based on Deep Learning and Lexical Analysis," Ninth Annual Conference of Japanese Association for Digital Humanities (JADH2019), Aug. 2019. (Osaka, Japan).
[21]LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105.
[22]Simonyan K, Zisserman A. Very deep convolutional networks for large scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[23]Liu W, Anguelov D, Erhan D, et al. Ssd: Single shot multi-box detector[C]//European conference on computer vision. Springer, Cham, 2016: 21-37.
[24]Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1-9
[25]He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[26]Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708.
[27]Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 28182826.
[28]Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017.
[29]Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1251-1258.