=Paper=
{{Paper
|id=Vol-2760/paper5
|storemode=property
|title=Privacy-Preserving Monitoring System with Ultra Low-Resolution Infrared Sensor
|pdfUrl=https://ceur-ws.org/Vol-2760/paper5.pdf
|volume=Vol-2760
|authors=Miyuki Ogata,Shogo Murakami,Takumi Mikura,Ikuko Eguchi Yairi
|dblpUrl=https://dblp.org/rec/conf/ijcai/OgataMMY20
}}
==Privacy-Preserving Monitoring System with Ultra Low-Resolution Infrared Sensor==
https://ceur-ws.org/Vol-2760/paper5.pdf
Privacy-Preserving Monitoring System with Ultra Low-Resolution Infrared Sensor
Miyuki Ogata1, Shogo Murakami1, Kimura Takumi1, Ikuko Eguchi Yairi1
1
Department of Information and Communication Sciences, Sophia University, Tokyo, Japan
{ogata | murakami | kimurat}@yairilab.net, i.e.yairi@sophia.ac.jp
undiscovered for long stretches of time. This phenomenon
Abstract has garnered attention of the public as the known incidents of
Action monitoring systems used in households lonely deaths continue to grow [Cabinet Office, 2016]. Full
provides vital information for health monitoring time supervision and assistance of the elderly population is
particularly with aging residents. While visual needed to prevent these kinds of deaths however; it often
inputs such as information provided by cameras comes at a high price. Care facilities have the advantage of
can recognize the actions and position of a subject offering 24-hour care, but the economic burden becomes
with high accuracy, they are not widely accepted higher the longer the patient remains. Conversely, it is
due to privacy concerns. This paper proposes a impossible for a single caregiver to tend to a senior resident
posture classification method with the use of a at all times thus being inefficient in preventing sudden
low-resolution thermal sensor. The sensor aims to incidents. For these reasons, demand for a reliable monitoring
protect the subject’s privacy by capturing visual system to help improve health care has grown.
input in the infrared spectrum as well as having a Supervision with a monitoring system through the use of
low spatial resolution of 8x8 pixels. We consider IoT would present a more adequate and cheap solution to
a simulation which recreates the experimental human alternative. Once installed, IoT devices can be used at
environment and produces data for this postural- all times in order to ensure the safety of the user. The two
behavioral problem. The validity of this method is methods that are currently available are wearable sensors and
checked by considering 3 postures; standing, visual sensors.
sitting, and laying down and examined using a Improvements in artificial intelligence and the spread of the
classifier on simulated data. Additionally, we internet has made behavior analysis more accessible.
explore optimal position and angle of the sensor as Wearable sensors focus on obtaining behavior analysis at a
well as the effects of color depth on accuracy. In low cost, low energy consumption and provides data
our results we achieve over 93% classification simplicity [Mukhopadhyay, 2015]. Devices such as
accuracy by color conversion of the infrared array smartphones and smartwatches can detect falls, share its
sensor image and successfully decreased loss due location, or record cardiac beats [Murad et al., 2017; Won-
to displacement by DCNN. We discover higher Jae et al., 2014; Najafi et al., 2003]. In general, the tri-axial
accuracies are achieved when the sensor is located accelerometers are used to analyze the movements and
50cm below the subject’s height with a tilt angle position of the device wearer. Over recent years there has
of ±2°. been a rapid development in this technology, yet issues
surrounding this technique still remain. These include limited
1 Introduction battery life, poor comfort, extended period of wear, and
Japan, like many other countries, is said to have a rapidly irregular wearing. The latter is the most problematic in the
aging society with more than 26.6% of its population being elderly population as many suffer from illnesses such as
over the age of 65 [Statistics Bureau, 2019]. As an aging dementia, Alzheimer or simply becoming forgetful and as
society increases, so does the incidence of suffering from such, overlooking to wear these devices.
medical conditions which includes but is not limited to; Sensors can also be embedded into daily items for
cardiovascular, musculoskeletal, and cognitive disorders, as monitoring and activity recognition purposes. A particular
well as other chronic diseases. These diseases often require object’s use frequency can be recorded to detect abnormal
around the clock supervision traditionally provided by activity. For example, a sensor in a cane would register an
caregivers or family members. In addition to this, Japan has abnormal movement if the cane were to suddenly fall to the
the issue of “kodokushi” or “lonely deaths” which refers to ground [Vahdatpour et al., 2010]. While this method protects
the phenomenon of people dying alone and often remaining the user’s privacy, monitoring is limited to scenarios in which
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
26
these items are being handled. Multiple devices can be Previous studies for detection, counting, and tracking of
tracked to increase broadness, leading to increased people have been performed with an 16x16 infrared sensor
complexity and high costs. for indoor monitoring [Berger and Armitage, 2010].
Visual sensors can capture high resolution images of the Recognition of hand motion direction has also been
user’s daily activity. This method contains information of investigated using a 4x4 infrared sensor [Wojtczuk et al.,
multiple events concurrently, which allows for a rich data 2011] however this extremely low resolution would not be
behavior analysis. This in turn outputs a very accurate suitable for more complex visual recognition.
recognition of behaviors. In an ever-growing digital age, Activity monitoring using an 8x8 infrared sensor has
being recorded constantly can be a deterrent for many users been successfully researched before [Tao et al., 2018].
in the form of privacy concerns. To eliminate behavioral However, these studies have recorded data in optimal
limitations and physical stresses for the user, infrared sensors conditions only. External factors are often not considered in
can be employed in monitoring systems. These sensors only the sensor and systems ability to perform. Effectiveness of
detect temperature information, making its use and these devices could be limited by both the location of
installation simple while keeping privacy breaches to a installation and installation angle. The contribution of color
minimum [Okada and Yairi, 2013]. depth in the accuracy of classification is also not thoroughly
Infrared (IR) monitoring devices are categorized into studied in the field of monitoring systems. Additionally, we
those using single beam sensors and those using an array of need to consider the difficulty regarding the acquisition of
sensors. While single beam IR sensors sense temperature at a learning data to increase classification accuracy. The
single point, IR array sensors are comprised of multiple investigation of participants under different conditions such
single beams to obtain deeper spatial information. IR array as room size, room temperature, body type, and posture
sensors obtain spatial information in three dimensions and would require extensive preparation and could lead to
receive light magnitude within a specified region to make inaccuracies. Collecting data from actual participants in this
posture estimations of users [Hayashida et al., 2017]. Current way would be unrealistic due to time limitations and the high
concerns with IR sensors include the accuracy and sensitivity costs involved.
of array sensors to changes in background temperature,
installation position, and installation angle. As of today, 3 Proposed Method
studies have only recorded data in optimal conditions.
Consequently, the impact of variances in position and angle 3.1 Device Setup
of IR sensor is not known. The device shown in Figure 1 is used for data collection of
This paper focuses on the classification accuracy of three posture classification. This device is comprised of an infrared
postures with the use of DCNN and a single 8x8 infrared array sensor Grid-EYE by Panasonic mounted on a series of
array sensor. Experimental and simulated data is used to single-board computer, Raspberry Pi3 Model B. The Grid-
investigate its effectiveness as a privacy preserving EYE sensor has an output of 8x8 data for surface
monitoring system. temperatures detected in the observed space. This data can be
visually represented as an 8x8 greyscale image. The sensor
2 Related Works detects temperatures from 0°C to 80°C with a temperature
The use of computer vision to assist in personal well-being resolution of 0.25°C. Its viewing angle is 60° with a sampling
and reduce incidents at home has increased in demand over rate of 10fps [Panasonic, 2016].
the last decade. Studies focused on wearable devices and
high-resolution monitoring systems have been widely
explored. In contrast, reports on privacy preserving vision-
based monitoring systems are still quite limited. Low-
resolution thermal sensors can detect temperature with the
use of a small array of infrared sensors. This produces a
spatial distribution of temperature which can be represented
in the form of low-resolution images. While cameras are able
to distinguish an individual’s features, infrared sensors can
only capture the outline or shape of the human body, making
identification of an individual difficult. The use of low spatial
resolution further decreases the image resolution making
identification nearly impossible. Utilization of this sensor is
the most suitable for homes as it provides a more comfortable Figure 1: Action recognition device with 8x8 infrared sensor
experience by being small, unobtrusive, and cheap while
protecting an individual’s privacy.
27
3.2 Deep Convolutional Neural Network (DCNN)
An effective method for obtaining high accuracy in image
recognition is with the use of Deep Convolutional Neural
Network (DCNN). In this study we implement DCNN to
extract features from infrared image data in order to train the
posture classifier. The network contains five layers: an input
layer, two convolutional layers, a fully connected layer, and
an output layer [Kimura et al., 2019]. Max pooling was
implemented in the convolution layers [Yang et al.,2015].
Dropout was used to prevent over learning and Adam
optimizer was employed to update parameter weighting to
optimum levels [Kingma and Ba, 2014].
3.3 Experimental Setting and Experimental
Dataset
Our experiment was conducted on 3 subjects to evaluate the Table 1: Classification result and evaluation from infrared sensor
performance of DCNN on posture classification. The position images by DCNN
of two subjects; one male (age 24) with a height of 170cm
and one female (age 20) with a height of 160cm were 8x8 colored images, which will then be pre-processed to
recorded in a 9.5m2 Japanese-style room with ambient input into the DCNN. The conversion into different color
temperature of 13°C. The position of the third subject; one spaces such as RGB, HSV, CIE XYZ, and CIE Lab can be
male (age 22) with a height of 170cm was recorded in a 20m2 used to increase feature recognition [Rachmadi and Purnama,
room with ambient temperature of 11°C. In all three set-ups, 2015]. Other studies have also used a combination of color
the sensor was placed 140cm form the ground so that the spaces to create an efficient face recognition system
subjects’ entire body could be observed. Subjects were told [Kurylyak et al., 2009]. Since the mentioned studies only use
to stand, sit, or lie down remaining within 1m to 3m from the color conversion on high resolution image data, its effects on
sensor so as to remain within the sensor viewing range. A low resolution image data are explored in this study. The
total of 14,983 frames worth of data was recorded, a most effective color conversion for this subject will be chosen
randomized 10% of which was used to train the classifier and through the processing of data into various color spaces and
the remaining 90% used for testing. Accuracy was found to comparing results.
plateau at around 100 epochs, consequently it was set as such.
Since the sensor does not discriminate between subjects,
posture classification result and accuracy evaluation are the
combined dataset from the three subjects. The results are as
shown in Table 1.
For real-world use, resident monitoring systems should
be able to detect sudden incidents such as slips or falls,
therefore the F-measure should be at or above 90%.
Experimental results showed an average F-measure of 87%.
By observing the classification results we found that incorrect
categorizations were predominantly in postures going from
standing to seated, and from seated to laying down.
3.4 Color conversion on Classification Accuracy
The values from the infrared sensor were directly inputted
into the DCNN. However, due to the quantity of erroneously Table 2: Classification evaluation from 8x8 RGB converted
categorized data, it can be hypothesized that without further images.
processing, an accurate feature extraction cannot be achieved.
A previous study by Ito [2018], reported on on-board camera The 8x8 greyscale data image were converted into color
and deep learning for pedestrian crossing detection. In it, they to evaluate classification accuracy. Color spaces as
converted greyscale data into colored images with the determined by the CIE (Commision Interon acznationale de
addition of edge processing and were able to significantly l’Eclairage) such as RGB, CIE XYZ, CIE L*a*b* were
increase accuracy [Ito et al.,2018]. Similarly, our DCNN is assessed. For the RGB color space, the greyscale image is
able to handle multiple channel inputs, including color divided into three image layers each representing R, G, B
images. The 8x8 infrared sensor values can be converted into which is fed into the DCNN. The input layer to the DCNN is
28
and for the walls are preset as well. After the simulator is turned on, the sensor emits a
traceable ray of light. When the ray reaches the human model, it measures and records
that temperature If it fails to reach the human model, the room temperature is recorded.
The human model has three types of postures namely, standing, seated or horizontal.
3.2 Evaluation of Data Learning using Pseudo-Data
converted from 1 channel into 3 channels. Temperature data low-resolution nature of the sensor both real data and
was mapped such that lower temperature values were shown simulated
Actual IR image data data areparticipants
from nearly indistinguishable
and pseudo-image from eachthe
data from other.
simulator are
in blues and higher values were shown in reds. This method A minor
shown and compared difference
in Figure can be perceived
4. Production in thedata
of simulator areamade
surrounding
use of participant
was repeated for both CIE XYZ and CIE L*a*b* with the use data andthe
realindividual in the
space data from real2.3.
Chapter data where
Figure temperature
4 is color-coded gradient
to show red portions
of OpenCV (Open Source Computer Vision Library) as high-temperature
appears as regions
a resultand
of blue
heatportions as low-temperature
transfer. In the simulated regions.
data Ita can be
developed by Willow Garage Inc. observedclearer
that the temperature
human portion split
of bothcan
databe
areappreciated
almost indistinguishable
between and the that re-
producibility is high.
The same test-train ratio as in the previous section was human model and background.
used to train the classifier. The classification and evaluation
process were repeated 100 times to assess their accuracy. 1m 2m 3m
Classification accuracy of 93% was achieved by XYZ; 91%
was achieved by L*a*b*. The best results were attained by
RGB color conversion as shown in Table 2. Real data
4 Experimental and Simulated Dataset
It is hypothesized that a simulated dataset can be created
without the need of human participants in order to increase Simulated
the volume of dataset for learning. A simulation that produces data
computer-generated data was constructed using Unity, a
cross-platform game engine with pre-installed libraries for Figure 2: Comparison of sensor data and simulated data for
physical functions. Its versatility for creating the requiredFig. 4. Comparison between real datastanding
from thesubject
sensing device and simulated data from the
learning data generator.
models as well as altering their conditions such as; modeled
humans’ height, physique, position, sensor tilt, height, and In our study, 5000 simulated images were used to train the
room size was the main reason for which Unity was selected. DCNN posture classifier and experimental data was used to
test the classification accuracy. This method achieved a 70-
4.1 Assessment of Experimental and Simulated 80% classification accuracy which is too low for practical use.
Datasets Causes for wrong categorization can be due to the heat
The Grid-EYE sensor uses 64 elements to detect temperature transfer from the body to surroundings or sources of heat such
through the measurement of emitted infrared light from as light, which increases the temperature of the participants’
objects. Furthermore, due to dissipation of heat and surrounding area. This background noise is not as prevalent
background noise, it was noted that increased distance from in the simulated data. Elimination of background heat from
sensor decreases the precision of temperature measured. In experimental data should decrease errors. Background data
order to replicate this, a virtual sensor was added to the elimination was applied to both sets of data; using the
simulation using ray tracing from point of observation. The simulated data for training and real data for testing. Results
use of ray tracing helps locate the human model and are as shown in Table 4 were F-measure of all categories
accurately recreates the physical occurrence caused by increased to over 90%.
distance.
Typically, data collected from simulators require highly
detailed human models to reproduce a real-life scenario.
However, since this study utilizes a low-resolution sensor
device a simple 3D model can replicate the experimental
setup. The human modeled was a rudimentary model
composed of legs, arms, a torso and a head. Temperature
distribution for each body part is set separately so as to
closely resemble the temperature distribution of a real human.
The room size and walls temperature distribution were set to
be analogous to the one present in the experimental setup.
The human model is able to be positioned in either standing,
seated, or laying down and was placed in random locations
between 1m to 3m away from the sensor.
4.2 Training and Results of Datasets
Infrared image data from participants and simulated data in Table 3: Classification result of real data by learning from
standing position at different distances is shown in Figure 2. simulated data
The figure is color coded to display areas of high temperature
in reds and areas of low temperatures in blues. Due to the
29
To investigate effects of external factors on classification accuracy, installation height
To investigate effects of external factors on classification accuracy, installation height
and tilt were analyzed in chapter 4. It was uncovered that accuracy peaks when the
and tilt were analyzed in chapter 4. It was uncovered that accuracy peaks when the
sensor is installed 50cm under model height, and that sensor tilt has an unneglectable
sensor is installed 50cm under model height, and that sensor tilt has an unneglectable
impact on accuracy. Currently, the most urgent concern is how to deal with installation
impact on accuracy. Currently, the most urgent concern is how to deal with installation
in abnormal conditions since adaptability of the monitoring device is extremely low.
in abnormal conditions since adaptability of the monitoring device is extremely low.
(F-measure)
(F-measure)
1.000
1.000
0.980
0.980
0.960
0.960
0.940
0.940
0.920
0.920
0.900
0.900
0.880
0.880
0.860
0.860 150cm 160cm 170cm
0.840 150cm 160cm 170cm
0.840
0.820
0.820 (cm)
170 160 150 140 130 120 110 100 90 80 70 60 50 (cm)
170 160 150 140 130 120 110 100 90 80 70 60 50
Figure 3: Person heightforwith
Fig. 5. Accuracy varying
various sensor position
sensor altitudes.
Fig. 5. Accuracy for various sensor altitudes.
(F-measure)
1.000(F-measure)
1.000
0.950
0.950
0.900
0.900
Table 4: Classification result of background removed real data by 0.850
0.850
learning from simulated data 0.800
0.800
4.3 Evaluation of Height and Angle of installation 0.750
0.750
-5 -4 -3 -2 -1 0 1 2 3 4 5 (degree)
-5 -4 -3 -2 -1 0 1 2 3 4 5 (degree)
Thus far, an accuracy of 90% has been achieved by thermal Figure 4: Vertical tilt angle for sensor
Fig. 6. Accuracy for vertical sensor angle.
sensor being positioned at 140cm off the ground. For actual Fig. 6. Accuracy for vertical sensor angle.
applications, the heights of users differ from individual to 5 Discussion
individual. Additionally, changes in sensor position or tilt can
occur due to external causes such as being shaken during an 5.1 Household Implementation
earthquake or being accidentally bumped into. We assessed For successful commercial use, the supervising system must
the sensors’ performance with different installation be easy to install and adaptable to buyers’ specification. The
conditions. device used in this experiment can be attached to an indoor
Using the simulation, sensor height was varied from room wall, placed at 50cm below clients’ height. If surface is
heights of 50-170cm in step increments of 10cm. Using the bumpy, blocked or angled it may need further adjustments.
data simulator, the effects were explored on human models Ease of use if another aspect of concern for monitoring
of 150cm, 160cm, and 170cm tall. Results shown in Figure 3 devices. In our case, the unobtrusive nature of the infrared
revealed classification accuracy on postures reached peak sensor device lends to the forgetfulness of seniors, once set
accuracy when sensors were installed at 50cm below the and installed there is no need for users to interact with the
users’ height. device.
The influence of tilt angle of sensor on classification While our system successfully detects three basic human
accuracy was also studied. Using a model height of 170cm postures (standing, sitting, and laying down) falls are not
and sensor set at its optimum 120cm elevation tilt was included. It is well known that fall action recognition is
explored between the ranges of –5° and 5° in incremental imperative in health monitoring for elders as quick response
steps of 1°. Positive angle describes sensor tilted upwards; is crucial in death prevention. To further differentiate actions
negative angle describes sensor tilted downwards. As such as sitting down versus sitting up and laying down versus
upwards tilt is increased F-measure tended to decline. The falling from three basic postures, we need to consider
same trend occurs when downwards tilt is applied, however temporal dimension in our dataset. We propose the addition
F-measure seems to severely drop after –1°, as seen in Figure of temporal feature extraction alongside spatial feature
4. F-measure is sustained over the value of 90% between the extraction for classification of actions as shown in Figure 5.
tilt ranges of –2° and 2°.
Protection of privacy is a serious concern when dealing 5.2 Temporalization of Action Recognition
with monitoring devices. Posture classification used in this Integration of Recurrent Neural Networks (RNN) can provide
research rely heavily on temperature distribution captured by us with the temporal feature maps of input data. RNNs
the 8x8 infrared sensor. Due to low resolution of the image, exhibits dynamic temporal behaviors as they can process
information received in each pixel is of vast importance. Any sequences of input and recognize patterns. The addition of
loss or noise could negatively affect DCNN ability to temporalization in our recognition system should provide us
categorize. Due to tilt, a full row of pixels might be shifted or with time dependent features extracted from the dataset to
completely evaded, preventing the full capture of the persons recognize actions.
posture. This loss of information will cripple the system’s Long Short-Term Memory (LSTM) is an artificial RNN
ability to classify postures correctly. which has been typically used in speech recognition and
30
Figure 5: Illustration of proposed human action recognition.
multi-language processing. LSTM in combination with Low image pixel dimension means that any cropping will
CNNs has been used in automatic image captioning, weather lead to significant loss of information.
forecast, and emotion recognition. It is our hypothesis that it
can also be used for temporal feature extraction in low 6 Conclusion
resolution sensor data. In this paper we explored the performance of infrared array
One of the difficulties in training an RNN for action sensor in resident monitoring system. Through the use of 8x8
recognition involves time frames. The time required for an sensor image we managed to yield over 90% accuracy for
individual to perform movements whether from standing to human posture classification. We analyzed data noise created
laying down or standing to falling can vary greatly. This by external factors on sensor tilt and position. We concluded
variation depends on factors such as age group, level of
that tilt angle within ±2° and a position of 50cm below
mobility, and health status or previous injuries. A study subjects’ height returned the highest accuracy. While height
recorded bed rising time (from supine to sitting position) variation made F-measure decrease by a maximum of 10%,
taking an average of 2.5s for adults, 4s for seniors in
tilt variation can decrease F-measure by over 25%. This
congregate housing, and 10s for seniors in skilled nursing
highlights the importance of proper positioning and tilt of
facilities [Alexander et al., 2000]. Moreover, external factors
sensor according to room size and users’ height. These are
such as type of fall and trying to stop a fall by holding on to
strong variables and are key to record movement and
side-rails, canes or other objects can also change the time optimize accuracy.
frame of the fall. Detecting for such inconsistent and irregular Additionally, we extended our study by introducing
falls might be difficult to train for, as they could be outliers
simulated data and found that it is a viable complementary
in the sample.
way to increase data sample size. We believe our work is the
5.3 Personalization and Issues first to apply a simulation model to increase data for low
resolution monitoring in the field of action recognition.
Classification of actions can be divided by age groups, with For further research, we can improve the data simulator
shorter time frames dedicated to younger/healthier seniors; and learning algorithm by including temporal feature
and longer time frames for older weaker seniors. extraction for action recognition and application in real
Alternatively, a broader time frame can be set to encompass environment. As shown in Figure 5 we hypothesize simulated
all age groups. A drawback for this method is that time frames data can be used to assist on data training without the need of
for other movements might be overlap. lots of real data recordings. The real data will undergo
The monitoring system can be further personalized by background removal to broaden our monitoring system
inputting real data from user detected by thermal sensor capability.
device. However, actions such as sitting, standing, and laying
down can cause strain when performed repeatedly by senior
citizens. Moreover, actions such as falling are potentially Reference
dangerous when repeated, making its data collection not [Alexander et al., 2000] Neil B. Alexander, Julie C.
feasible. Instead, as shown in Chapter 4.1 the use of data Grunawalt, Scott Carlos, and Joshua Augustine. Bed mobility
generating simulator can be used to increase dataset for these task performance in older adults. Journal of rehabilitation
actions lessening the risk of injury from its users. research and development 37(5):633-638, September 2000.
The degree to which this system can be personalized is
still to be determined. While 8x8 sensor preserves the user [Berger and Armitage, 2010] Martin Berger and Alistair
information privacy and is faster to process, it also brings Armitage. Room occupancy measurement using low-
forth issues due to its low resolution. Hyper low-resolution resolution infrared cameras. IET Irish Signals and Systems
sensors inherently have the disadvantage of containing less Conference (ISSC 2010), pages 249-254, June 2010.
information compared to their high-resolution counterparts.
31
[Cabinet Office, 2016] Cabinet Office. Annual Report on the [Panasonic, 2016] Panasonic. Infrared Array Sensor Grid-
Ageing Society: FY 2019. Cabinet Office Government of EYE specification sheet.
Japan, June 2019. http://www.farnell.com/datasheets/2058257.pdf?_ga=2.187
765294.767814663.1555052764-1882353189.1555052764
[Hayashida et al., 2017] Akira Hayashida, Vasily Moshnyaga, created, May 2016. Last visited Jan 2020.
and Koji Hashimoto. The use of thermal ir array sensor for
indoor fall detection. In 2017 IEEE International Conference [Rachmadi and Purnama, 2015] Reza Fuad Rachmadi and I.
on Systems, Man, and Cybernetics (SMC), pages 594-599, Ketut Eddy Purnama. Vehicle color recognition using
December 2017. convolutional neural network. Computing Research
Repository Journal (CoRR), October 2015.
[Ito et al.,2018] Toshio Ito, Yuki Takata, and Mohd Hafiz
Hilman bin Mohammad Sofian. Crossing Pedestrian [Statistics Bureau, 2019] Statistics Bureau. Statistical
Detection Using Deep Learning by On-board Handbook of Japan 2019. Ministry of internal Affairs and
Camera. Transactions of Society of Automotive Engineers of Communications, pages 8-16, May 2019.
Japan, 49(5), September 2018.
[Tao et al., 2018] Lili Tao, Timothy Volonakis, Bo Tan,
[Kimura et al., 2019] Takumi Kimura, Shogo Murakami, and Yanguo Jing, Kevin Chetty, and Melvyn Smith. Home
Ikuko Eguchi Yairi. Privacy-Preserving Resident Monitoring Activity Monitoring using Low Resolution Infrared Sensor
System with Ultra Low-Resolution Imaging and the Array. Computing Research Repository Journal (CoRR),
Examination of Its Ease of Installation. The Japanese Society November 2018.
for Artificial Intelligence 33rd annual conference, pages
4D3E201-4D3E201, June 2019. [Vahdatpour et al., 2010] Alireza Vahdatpour and Majid
Sarrafzadeh. Unsupervised discovery of abnormal activity
[Kingma and Ba, 2014] D. Kingma and J. Ba. Adam: A occurrences in multi-dimensional time series, with
Method for Stochastic Optimization, International applications in wearable systems. In Proceedings of the 2010
Conference on Learning Representations, Dec 2014. SIAM International Conference on Data Mining, pages 641-
652. Society for Industrial and Applied Mathematics, May
[Kurylyak et al., 2009] Yuriy Kurylyak, Ihor Paliy, Anatoly 2010.
Sachenko, Amine Chohra, and Kurosh Madani. Face
detection on grayscale and color images using combined [Wojtczuk et al., 2011] Piotr Wojtczuk, Alistair Armitage, T.
cascade of classifiers. Computing = Комп’ютинг 8(1), pages David Binnie, and Tim Chamberlain. PIR sensor array for
61-71, 2009. hand motion recognition. In Proceedings of the 2nd
International Conference on Sensor Device Technologies
[Mukhopadhyay, 2015] Subhas C. Mukhopadhyay. Wearable and Applications, pages 99-102, August 2011.
sensors for human activity monitoring: A review. IEEE
sensors journal, 15(3):1321-1330, March 2015. [Won-Jae et al., 2014] Yi Won-Jae, Oishee Sarkar, Sivisa
Mathavan, and Jafar Saniie. Wearable sensor data fusion for
[Murad et al., 2017] Abdulmajid Murad and Jae-Young Pyun. remote health assessment and fall detection. IEEE
Deep recurrent neural networks for human activity International Conference on Electro/Information Technology,
recognition. Sensors Journal, 17(11):2556, November 2017. pages 303-307, June 2014.
[Najafi et al., 2003] Bijan Najafi, Kamiar Aminian, Anisoara [Yang et al.,2015] J.B. Yang, M.N. Nguyen, P.P. San, X.L.
Paraschiv-Ionescu, François Loew, Christophe J. Bula, and Li, S. Krishnaswamy. Deep Convolutional Neural Networks
Philippe Robert. Ambulatory system for human motion On Multichannel Time Series For Human Activity
analysis using a kinematic sensor: monitoring of daily Recognition. In Proceedings of the 24th International Joint
physical activity in the elderly. IEEE Transactions on Conference on Artificial Intelligence (IJCAI), pages 3995–
biomedical Engineering 50(6):711-723, June 2003. 4001, Buenos Aires, Argentina, June 2015.
[Okada and Yairi, 2013] Ryotaro Okada and Ikuko Yairi. An
indoor human behavior gathering system toward future
support for visually impaired people. In Proceedings of the
15th International ACM SIGACCESS Conference on
Computers and Accessibility, article 36, October 2013.
32