Personal hygiene is often used to measure functional independence, which is how much support someone requires to perform self-care. By extension, this is often used in the monitoring of (early-stage) dementia. Current technologies are based on either audiovisual or wearable technologies, both of which have practical limitations. The use of (NLOS) radio-frequency based human activity recognition could provide solutions here. This paper leverages the 802.11n channel state information to monitor diferent shower-related activities (e.g. washing head or body, brushing teeth, and dressing up) and the degree to which some of these can be monitored, as well estimating diferent water pressures used while showering for multiple locations in the apartment. Wavelet denoising is applied for filtering and a convolutional neural network is implemented for classification. Results imply that for coarse-grained activity recognition, an 1-score of 0.85 is achievable for certain classes, while for ifne-grained this drops to 0.75. Water pressure estimation ranges from 0.75 to 0.85 between fine-grained and coarse-grained, respectively. Overall, this paper shows that channel state information can be successfully employed to monitor variations in diferent shower activities, as well as successfully estimating the water pressure in the shower.
• Computer systems organization → Sensor networks; • Networks → Wireless local area networks; • Human-centered computing → Ubiquitous and mobile computing.
Personal hygiene is an important aspect in elderly care and even
more so for those sufering from (early-stage) dementia. It is an
important determinant of the level of support someone requires to
perform self-care, which is part of the functional independence in
activities of daily living (ADL) [
Unlike the aforementioned technologies, radio frequency-based
technologies (unobtrusive sensing) do not have to be put inside
the shower itself, are more robust (e.g. do not need to be worn
and cannot be forgotten) and are likely more privacy-aware, as the
data is not easily interpretable by humans, since it requires more
complex (pre)processing. This enables ADL monitoring [
This paper leverages channel state information in a non lineof-sight (NLOS) environment to monitor the degree of shower behaviour on multiple locations in an apartment. This degree comes in the form of several shower-related activities, but performed in two diferent ways: one regular showering (as a person would), the other is by acting as if showering is troublesome (slow, painful, troublesome). Additionally, the activities of (un)dressing, drying of the body and brushing of the teeth are considered in their regular form. Results suggest that RF-based sensing can be used to investigate the aforementioned activities with an 1-score score over 0.85, while also achieving a comparable 1-score score for water pressure estimation, depending of the location of the receiver.
This paper first discusses the related works, which include works which inspired this research, as well as current solutions using other technologies. After that the data acquisition is discussed, as well as an overview of the dataset, to encourage fellow researcher to replicate this work and contribute to the presented dataset. During data acquisition, the variables are also discussed and how these are chosen. Following this, the actual methodology is discussed, which includes the ground truth evaluation, signal processing and data analysis. In the following Results and Discussion section, the actual outcome is evaluated and the research questions will be answered. Finally, the paper concludes with a short summary of the results and suggestions for future work. 1.1
Currently, little to no research has gone into personal hygiene
detection under the shower, which is used in the monitoring of
(early-stage) dementia. The use of audiovisual, wearable of
infrastructural monitoring technologies has been proven [
The use of indirect (NLOS) radio-frequency based sensing could change this, as it reduced both the immediate privacy issues (video cameras under the shower) and the need for water-proof wearable devices and the wireless communication. Additionally, no infrastructural changes to a home are required, such as expensive shower heads, or tiles- and wall-mounted sensors. Due to the unobtrusive nature, RF-based sensing could prove to be useful in monitoring the variation to which shower activities are being performed, as well as combine these with shower information (e.g. water pressure). Multiple questions arise here in relation to the positioning of diferent receivers, but these will be discussed during the data acquisition and methodology.
The contributions of this paper are: • To show the extend to which variations of diferent activities can be identified and monitored under the shower • To show the extend to which diferent water pressures can be identified and monitored under the shower • To investigate the efect of diferent receiver placement on the accuracy of the aforementioned aspects 2
Channel state information is one of the most prominent RF-based sensing techniques. It gathers information regarding the signal multipath propagation between a transmitter and receiving antenna at the receiver side. Multipath propagation is a phenomenon caused by environmental influences on the signal, such as scattering, absorbing, and reflecting. These environmental influences include humans, and when monitoring the changes in the channel state information over time, the activities of them. Channel state information contains information on the phase and amplitude of the received signal. The combination of all these antenna pairs is collected in a channel state information matrix, which has the shape of ∗ ∗ , where is the number of receiving antennas, the number of transmitting antennas, and the number of subcarriers.
Research in activity recognition under the shower is fairly
limited. Lee et al. [
Zhang et al. [
Wang et al. [
The setup consisted of a multiple custom Gigabyte Brix IoT devices,
each consisting of an Intel Apollo Lake N34500 processor, 8GB
DDRL 1866 MHz memory, and a n Intel N Ultimate Wi-Fi Link 5300
as a network interface. For these experiments, one functioned as a
transmitter and the others as receivers. While the Linux CSI Tool
[
While multiple locations were considered for the transmitter, ultimately a setup was picked that consists of four receivers and a single transmitter (Figure 1). Here, the transmitter (red triangle) is located in the farthest corner from the apartment, where the actual Wi-Fi router would be located in this real-life setting. The first receiver is positioned in direct line-of-sight from the transmitter (∼ 7) as a zero-measurement (LR). The second receiver is placed in the fuse box compartment of the apartment (FB), positioning it skewed behind the shower (∼ 9.6), as it is close to the shower and the potential water pipes leading into it. The third receiver is located in the study room (SR), directly behind the the bath room. However, while the receiver is theoretically closer than receiver FB (∼ 8.5), the signal needs to propagate through more obstacles (e.g. walls and doors). The fourth and final receiver is placed in the bed (BR), which would theoretically result in the highest coverage (ignoring any obstruction). While it has the best propagation conditions (least overlap with the shower), it is the farthest away form the transmitter (13.9).
It should be noted that the data was collected over multiple days, with the apartment being lived in regularly. This means furniture may be moved around and there could be slight changes to the position of the receivers. Overall, this setup is more relevant than a ifxed laboratory setup, as it represents the actual use-case of such a system in the future. However, it was ensured no additional faucets (e.g. kitchen sink or dishwasher) were on while showering, as this was out of the scope for this research.
It is assumed that participants likely live alone or shower at times they are alone. However, it should be noted that there were two people in the apartment at all times during these experiments: the person not showering was always sitting at the kitchen table as far away as possible, minimizing the amount of movement on the signals. Cross-activity recognition among multiple people (e.g. one shower, one cooking) is not considered as part of this paper. Therefore, the the receiver in the living room is used to verify this has a minimum influence on the data collected by the other receivers. While concrete details on the participants cannot be given due to privacy concerns, it can disclosed that the two participants were a male and female, with a height diference of 15cm and a weight diference of 20kg. This shows the participants’ physiques were not similar. 3.2
For this research, 9 activities are considered (Table 1). These activities are all based inside the bathroom, e.g. activities occurring under or around the shower in healthy participants and patients sufering from (early-stage) dementia. Therefore, some activities are not necessarily bound to showering (e.g. washing head/face), but also to other activities related to personal hygiene (e.g. brushing teeth).
All activities are considered with four diferent water pressures: of, low, medium, and high. It should be noted that these pressures can be minimized into a binary problem, namely of and on (low, medium, and high combined). It is likely that the water pressure is a personal preference, rather than there being a correlation between personal hygiene and the water pressure. Therefore, the binary problem of the shower being on or of is most prominent.
The activities under the shower are performed in two manners: one in a regular/excited fashion, the other in a slow/demotivated fashion. This is done to see if a diferentiation can be made between either, in order to make it possible to monitor the degradation of personal hygiene in patients with (early-stage) dementia. For regular/excited, it is suggested the participants shower as they normally would, or in a very good mood. This could be diferent depending on the participant, as the definition of a very good mood and the results of such a mood difer per participant. For slow/demotivated, participants are asked to pretend like they are either physically restricted or in a very bad mood while showering.
Due to the sensitive of the data, no ground truth could be collected. Rather, participants are asked to follow a set of instructions (playing through a speaker in the shower) in order to have analogous activities in the same time slots. Between each activity is a moment for idling, which is used for both classification and to separate diferent activities.
No smart shower head was used, so the exact L/min is unknown and it is likely that per run, there is a slight diference between each. However, an attempt was made to replicate each shower based on the visual appearance and sound of the water beam coming out of the shower head. For Low, the shower should drizzle: barely any water should come through the shower head. Med is the setting at which there is just enough pressure in the shower head to result in a consistent stream. The last setting, Max, requires the shower to be on at full force. 4 4.1
For preprocessing, the first step is restructuring of the channel state information, which is a 4D-matrix (3 ∗ 3 ∗ 30 ∗ ) due to 3x3 MIMO and 30 subcarriers over time , into the shape of the input layer to the convolutional neural network, which has a shape of a 3Dmatrix ( ∗ ∗ ). For these experiments, the data was flattened into a 3D-shape with a depth of 1 ( = 1), namely 270 ∗ ∗ 1, as 3 ∗ 3 ∗ 30 = 270. This means that for every antenna pair, the subcarriers are stacked on top of each other (e.g. the first 30 rows are receiving antenna 1 and transmitting antenna 1, the second 30 rows are receiving antenna 1 and transmitting antenna 2, and so on). Afterwards, wavelet denoising was used to denoise the signal in MATLAB, after which the data was stored as a set of images, where each pixels is thus based on an absolutely value for a specific subcarrier in a specific antenna-pair ( ) at time ( ). 4.2
A convolutional neural network (CNN) is employed to for classiifcation, as it preserves the spatial and structural information of the channel state information. The CNN consists of three 2D convolution layers, with a 0.6-dropout after the first and third layer. Max-pooling, batch normalization and a leaky ReLU ( = 0.1) for the activation layers are applied after each layer. At the end, the outputs are flattened and go through a 160-neuron dense layer with the sigmoid activation, before reaching the final dense layer for classification (softmax). The model was trained for 250 epochs, with a batch size of 8. The learning rate started with an initial learning rate of 1 ∗ 10−4, with a decay rate of 0.95 every 50 steps. The split between the training and testing set used is 0.60 and 0.40 for the data of both users combined, respectively. 4.2.1 Activity classification. The first thing to consider is the activity classification in, regardless of the water pressure. For this, the shower-related activities (bold in Table 1) are combined over the diferent water pressures (including Of ). This is to give an indication whether or not activity classification is afected by water pressure. A diferentiation is made between all all activities and shower-related activities, where all activities include activities only happening during Of and where shower-related activities only include those while the shower is on. This is evaluated over the diferent receiver locations. 4.2.2 Water pressure estimation. Here the labeling is based on the water pressure. All data is combined over the diferent activities depending on the water pressure. At the lowest level, the binary problem of shower On/Of is considered by clustering everything other than Of as On. Afterwards, this is turned into a more finegrained problem where an attempt is made to diferentiate per individual water pressure. For the classification of water pressure, only the shower-related activities are considered. 4.2.3 Receiver positioning. Finally, the activity classification and the water pressure estimation are compared between the diferent receivers (Figure 1). The most optimal location for these experiments will be discussed, as well as the worst performing one. For this, only the shower-related activities are used for both activity and water pressure recognition. Receiver LR is omitted, as it was used as a zero-measurement (no major movements happening in the living room).
(a) 5.1.1 Activity classification. Figure 2 presents the classification as a confusion matrix for all activities for = , = 0, as all activities are important for the activity recognition part. While Figure 2 shows the normalized confusion matrix over 10 runs, the average 1-score with standard deviation over these 10 runs are used as a metric when discussing specific classes. The overall 1-score for all activities is 0.74 ± 0.05. For the case of all activities, 4 out of 9 classes have an 1-score over 0.85 (BT, DU, DB, WHF-E) and 3 more an accuracy over 0.75 (WHF-S, WBL-E/S). The only exceptions here are idle (I ) 0.4 ± 0.0 and undress (UD), with an 1-score of 0.58 ± 14 and 0.03 ± 0.09, respectively.
For undress, this is likely due to the limited data, as the activity only lasts for 30 seconds. This causes only 40 data fragments after preprocessing. As a comparison, all other classes have at least 110 data fragments, with the showering activities (WHF-E/S,WBL-E/S,I ) over 330. Additionally, undressing takes place at approximately the same location as brushing teeth.
For Idle it is a bit diferent, as it has a comparable number of data frames after preprocessing, namely 334 data frames. Idle is mostly confused with brushing teeth (0.13 ± 0.01) and washing of the both the head and body (in the range of 0.11) for both slow and excited, likely due to these being activities involving minor body movements. Minor body movements are a part of idling, as participants cannot be expected to stand completely still in the shower (e.g. getting water in the eye or uncomfortable positioning).
Another observation is that there are darker squares in the confusion matrix around the two diferent performances (excited and slow) of Washing body and legs and Washing hair and face. For Washing body and legs and Washing hair and face, the 1-score is 0.77 ± 0.15 − 0.81 ± 0.07 and 0.80 ± 0.10 − 0.87 ± 0.07, respectively. The false-positives and true-negatives are in the range of 0.15, which is likely due to it being the same activity being performed with variation: sometimes an optimistic interpretation of slowly washing is close to a pessimistic interpretation of excited washing. Overall, this implies that an estimation can be made regarding the performance of washing, while there is a clearer distinction between washing the body and legs and washing the face. It is likely that the accuracy will drop when even more fine-grained activity recognition is considered (e.g. washing hair, face, upper body, and lower body). (a) (b) 5.1.2 Water pressure estimation. Figure 3 shows the confusion matrices for the classification of the four water pressures for = , = 0. Figure 3 includes the actual showering activities (such as idling and washing), namely − , −, − , −. Figures 3a,b are also normalized over the 10 runs.
For the binary problem of estimating whether the shower is on or of (Figure 3a), it is observable that this can be estimated with an 1-score of 0.99 ± 0 for the relevant activities (b).
For the individual shower pressures, the 1-score of Of is 0.97 ± 0.02 for classification with only the relevant activities. This is different for the other three: Low and Max have an 1-score in the range of 0.73 ± 0.19 − 0.79 ± 0.24 for relevant activities, which is lower than for Of . This indicates that it is harder to diferentiate between either. However, the lower 1-score is largely explained by investigating Med, with a 1-score of 0.48 ± 0.17: while Low and Max have false-positives and false-negatives between them, this is a minority compared to the false-positives and false-negatives between the two and Med, as can be observed in the confusion matrices by the darker colors and upon further inspection of the actual classification rates. This indicates a larger diference between Low and Max, but lesser so between the two and Med, which can be explained by small diferences in the shower sessions and the variation in the shower head: sometimes, medium may be exactly between low and maximum, but other times it may edge more towards either of the two. 5.1.3 Receiver positioning. Figure 4 shows the additional classiifcation performance for receivers FB (a,c), and BR (b,d). The top and bottom rows show the classification accuracy over 250 runs for = {0, 1} for activities and water pressure recognition, respectively. The classification is based on the relevant activities, as these are the best case scenarios. The classification for all activities performs worse, as is previously discussed in sections 5.1.1 and 5.1.2. The information for receiver SR can be found in Figure 2 and 3 for activities and water pressure, respectively.
It can be seen that receiver SR is the most prominent at identifying the activities (1-score of 0.74 ± 0.05) , while receiver BR is most prominent at water pressure estimation (0.88 ± 0.09). While both score comparable on activity recognition (1-score of 0.74 ± 0.05 and 0.75 ± 0.08 SR and BR, respectively), BR has fewer classes that need classification (5 against 9 for bedroom and study, respectively), meaning the actual performance of BR is lower. Thus, these results imply that the bedroom is better for estimating the shower pressures, likely due to less noise from the water and pipes and thus the better signal propagation conditions, but that SR is better activity recognition, likely do to the path crossing the actual activities.
The worst performing receiver is FB, with an 1-score of 0.25 ± 0.15, 0.90 ± 0.01, and 0.47 ± 0.28 for activity recognition, on/of, and individual water pressure estimation, respectively. The worse results could be explained by additional noise caused from the fuse box itself (more of interference from pipes running in and out), or that the most dominant signal is unafected by the shower: from the transmitter to FB, it only needs to penetrate one door before it reaches the receiver, as the fuse box compartment is open at the top. 6
The results indicate that it is possible to determine whether the shower is on or of with an 1-score of 0.99 ± 0.00 and individual shower pressures with an 1-score between 0.48 ± 0.17 and 0.79 ± 0.24 based over 10 runs. This implies that while it is challenging to detect certain individual shower pressures, it is feasible to monitor shower usage to a coarser degree. For activity recognition, an 1-score of 0.74±0.05 on average is found. However, it is possible to diferentiate between diferent levels (such as regular and slowed down movements) of shower-related activities (such as washing head or body) with an 1-score of 0.76 ± 0.17 and 0.93 ± 0.07. This indicates channel state information can be used to potentially monitor the personal hygiene for patients in self-care. Additionally, the results imply the optimal position to place the receiving receiver for the activity recognition is directly behind the shower, while for water pressure estimation it is recommended to put the receiver slightly skewed behind the shower for a more optimal propagation environment.
Future work would include validating the results in this paper through more participants, diferent locations for the transmitter, and diferent frequencies. Repeating the experiments with more (and diferent) participants for diferent settings (e.g. multi-floor houses or larger apartments) is a must to verify these results in on a larger scale and for these technologies to be adapted into real-life scenarios. While 5 GHz has proven useful in this experiment, 2.4 GHz could be viable due to its larger wavelength, which means it is better capable at penetrating walls. Additionally, either frequency could also be afected by water in a diferent way. The proposed implementation should also be tested in a real-time for fashion for real-time classification.