Introduction

The growing demographic of the elderly population has precipitated an increased incidence of falls [1], a leading cause of morbidity and mortality among this group [2]. The recent technological solutions for fall detection have emerged as a critical component in mitigating these risks [3]. Among these, video-based fall detection systems have shown significant promise due to their non-invasiveness and capability for real-time monitoring [4]. However, video surveillance in healthcare, particularly in homes and care facilities, raises significant privacy concerns [5]. This research aims to find a balance between ensuring safety through surveillance and upholding the right to privacy.

There has been a significant relationship between the efficacy of video-based fall detection and the imperative to protect individual privacy [6]. While effective in identifying falls, traditional approaches often overlook the privacy implications of constant video monitoring [7]. Possible solutions include avoiding video data, implementing basic obfuscation techniques [8], compromising effectiveness, or insufficient privacy [9]. Previous research has proposed various methods, including wearable devices [10][11][12][13] and environmental sensors [14][15][16], to circumvent the associated privacy issues. However, these alternatives fall short in accuracy and real-time response capabilities compared to video-based systems.

In response to these challenges, this paper proposes an innovative solution that retains the advantages of video surveillance while addressing privacy concerns. Our approach employs dynamic blurring, selectively obscuring identifiable features within video frames. Thus, individuals are anonymized without compromising the system's ability to detect falls. This method differs from existing solutions by offering a real-time, privacy-preserving mechanism that does not detract from the system's performance. Integrating EfficientNetB0 [17] for feature extraction and Long Short-Term Memory (LSTM) [18] networks for fall events ensures high precision in fall detection.

This research aims to develop a fall detection system that fulfills the need for efficient, realtime monitoring with the imperative of privacy preservation. Our objectives include designing and implementing a dynamic blurring technique within a video-based fall detection framework. Moreover, we also aim to evaluate this system's accuracy and privacy protection performance and demonstrate its applicability in real-world settings. This research can potentially contribute to the development of ethically responsible technological solutions in healthcare, particularly in the context of elderly care. This work seeks to pave the way for broader acceptance by addressing the privacy concerns associated with video-based monitoring. Moreover, deploying such systems enhances the safety and well-being of the elderly population.

Literature Review

The exploration of fall detection systems, particularly for the elderly, is an area of research that has seen substantial evolution over time. Dean et al. [19] 2006 implemented the first real-time fall detection system using a triaxial accelerometer. At that time, most traditional techniques centered around simplistic, mechanical solutions and gradually transitioned towards incorporating technology [20]. Among the earliest methods were basic alert systems, which relied on the user to trigger an alert manually in case of a fall [21]. While pioneering for their time, these systems were limited by their dependence on the users to activate the alarm post-fall, which could be compromised due to injury.

Advancements in technology brought in a new wave of methodologies, primarily categorized into sensor-based [14][15][16], wearable devices [10], [12], and video surveillance systems [22], [23][24][25], alongside other innovative approaches. Sensor-based systems often utilize accelerometers and gyroscopes to detect sudden movements or orientations indicative of a fall. Wearable devices, such as smartwatches [26], integrate these sensors and offer portability. However, sensor-based and wearable systems face challenges related to user compliance, discomfort, and the potential for false positives due to non-fall-related abrupt movements [27]. In contrast, video surveillance systems offer a less intrusive alternative, capturing a broader context of the individual's environment [28]. This method's appeal lies in its passive nature, requiring no active input or wearables from the monitored individuals. Despite these advantages, video-based systems have challenges [29]. High-quality video processing demands significant computational resources, and managing vast data volumes poses storage and efficiency concerns. Moreover, the critical issue of privacy infringement emerges, given the intrusive nature of continuous video monitoring [9].

Traditional algorithms such as Support Vector Machines (SVMs) [30] and Decision Trees [31] were widely employed in the early stages of machine learning applications for fall detection. These methods primarily relied on handcrafted features extracted from sensor data or basic video analytics, including motion vectors and silhouette shapes. Flow-based methods, particularly optical flow [32], were also prominent, enabling the detection of movement patterns by analyzing the apparent motion of objects, surfaces, and edges. While effective to a certain extent, these approaches faced limitations in handling the high variability and complexity of human falls. They often struggled to distinguish falls from other activities involving rapid movements, leading to high false alarm rates [33], [34]. Additionally, their dependency on manually crafted features restricted their adaptability, as these features might not generalize well across different scenarios.

The recent emergence of deep learning architectures like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) has changed many dynamics [35]. Advanced models such as ResNet [36], LSTM [37], and YOLO especially marked a leap forward in fall detection. CNNs, with their ability to perform automatic feature extraction, have proven particularly adept at analyzing spatial characteristics in video frames [38]. While RNNs and LSTMs excel in capturing temporal dependencies, it is crucial to understand the sequence of movements leading to a fall. YOLO [39], an object detection model, brought further advancements by enabling real-time processing. Despite their successes, the search for enhanced performance led to exploring hybrid methods that combine multiple deep learning models. For instance, integrating CNNs with LSTMs allows for the effective processing of video data both spatially and temporally, offering a better understanding of fall events [40]. These hybrid approaches [41], alongside innovative methods within deep learning frameworks, promise to address the dynamics of fall detection [42].

Recent advancements aim to address these privacy concerns while maintaining system efficacy. Techniques such as dynamic blurring and real-time anonymization have been explored to obscure identifiable features in video feeds. This can help safeguard individual privacy without significantly compromising detection capabilities. Despite these efforts, there is a gap in the literature concerning developing a system that seamlessly integrates high detection accuracy with robust privacy protection. Our contribution to this field addresses this gap by proposing a novel fall detection system that employs EfficientNetB0 for advanced feature extraction and LSTM networks for accurate temporal classification, complemented by a dynamic blurring mechanism to ensure privacy. This integrated approach promises high performance, as evidenced by optimal accuracy, recall, and precision scores. Moreover, it introduces a viable solution to the privacy concerns that have long shadowed video-based monitoring systems. By achieving this delicate balance, our research paves the way for the broader acceptance of videobased fall detection systems, ensuring the safety of the elderly population.

Methodology

This research presents a methodological framework to address the challenge of detecting falls through video surveillance while safeguarding their privacy. The foundation of the proposed approach is mathematical models and techniques to ensure precision, efficiency, and reliability. The proposed method integrates state-of-the-art EfficientNetB0 for spatial feature extraction and LSTM networks for temporal sequence analysis. Additionally, we introduce a dynamic blurring mechanism formulated to preserve privacy by selectively obscuring identifiable features within video frames. Figure 1. provides the overall architecture of the proposed approach.

Dataset and Processing

The dataset employed in this study was sourced from the UR Fall Detection Dataset [43], encompassing 70 sequences, of which 30 are fall events, and 40 represent activities of daily living (ADL). The fall events were captured using two Microsoft Kinect cameras, accompanied by accelerometric data, whereas the ADL events were documented using a single camera (camera 0) alongside accelerometer data. The accelerometric data was acquired through PS Move (60Hz) and x-IMU (256Hz) devices. The dataset is structured such that each sequence comprises depth and RGB images from both camera perspectives (parallel to the floor and ceiling-mounted), synchronization data, and raw accelerometer readings. Each video stream is archived separately as a sequence of PNG images. The depth data, stored in PNG16 format, necessitates rescaling to represent depth in millimeters (D) as follows accurately:

𝐷 𝑖 (𝑥, 𝑦) = 𝑉(𝑥,𝑦)⋅𝑆 𝑖 65535 (1)

Where 𝐷 𝑖 (𝑥, 𝑦) denotes the depth at position (𝑥, 𝑦) for the ith camera, 𝑉(𝑥, 𝑦) represents the pixel value at position (𝑥, 𝑦) in the PNG16 image, and 𝑆 𝑖 is the scale ratio for the i-th camera. The scale ratios are defined as 𝑆 0 = 6000 for fall sequences using camera 0, 𝑆 1 = 3640 for fall sequences using camera 1, and 𝑆 0 = 7000 for ADL sequences using camera 0. The preprocessing of video data involves a series of steps to prepare the frames for feature extraction. Initially, each video is accessed frame by frame using OpenCV's VideoCapture functionality. Subsequently, each frame is resized to a uniform dimension of 224 × 224 pixels to align with the input requirements of the EfficientNetB0 model. This resizing operation can be mathematically represented as a function R that maps the original frame dimensions to the target dimensions, preserving the aspect ratio and interpolating pixel values as necessary:

𝑅: ℝ 𝑤×ℎ×3 → ℝ 224×224×3(2)

Where w and ℎ denote the original width and height of the frame, respectively. After resizing, the frames undergo normalization to scale the pixel values to the [0, 1] range, facilitating more stable and efficient model training. The normalization process for a frame F can be defined as:

𝐹 𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑 = 𝐹 225(3)

This operation ensures that each pixel value in the frame is proportionally reduced to a decimal between 0 and 1, thus standardizing the input data for subsequent processing through the EfficientNetB0 architecture.

Feature Extraction Using EfficientNetB0

The feature extraction component of our methodology is built upon the EfficientNetB0 architecture, a cutting-edge CNN known for its scalability and efficiency. EfficientNetB0 uniformly scales the network's depth, width, and resolution, optimizing its performance across various constraints. EfficientNetB0 are its convolutional operations, which form the backbone of its feature extraction capabilities. A convolutional operation on an input image or feature map can be mathematically described as:

𝐹 𝑜𝑢𝑡 (𝑥, 𝑦) = ∑ ∑ 𝐾 ( 𝑏 𝑗= −𝑏 𝑎 𝑖= −𝑎 𝑖, 𝑗) ⋅ 𝐹 𝑖𝑛 (𝑥 − 𝑖, 𝑦 − 𝑗)(4)

Where Fout is the output feature map, Fin is the input image or feature map, K is the kernel or filter of size (2𝑎 + 1) × (2𝑏 + 1), and (𝑥, 𝑦) denotes the pixel coordinates. This operation is applied across the entire input feature map, extracting features through the weighted summation of pixel values within the kernel's receptive field. EfficientNetB0 also leverages batch normalization to enhance training stability and convergence. Batch normalization can be defined as:

𝐵𝑁(𝑥) = 𝛾( 𝑥−𝜇𝐵 √𝜎 2 𝐵 +𝜖 ) + 𝛽(5)

Where x is the input to the batch normalization layer, μB and σB2 are the mean and variance of the batch, respectively, γ and β are learnable parameters of the layer, and ϵ is a small constant added for numerical stability. Furthermore, EfficientNetB0 employs depthwise separable convolutions, a technique that reduces computational cost without sacrificing depth or expressivity. A depthwise separable convolution comprises two stages: depthwise and pointwise convolution. The depthwise convolution applies a single filter per input channel, and the pointwise convolution then combines the output channels using a 1×11×1 convolution. This can be represented as:

𝐷𝑊(𝑥, 𝑦, 𝑐) = ∑ ∑ 𝐾 𝑐 ( 𝑏 𝑗= −𝑏 𝑎 𝑖= −𝑎 𝑖, 𝑗) ⋅ 𝐹 𝑖𝑛 (𝑥 − 𝑖, 𝑦 − 𝑗, 𝑐) (6) 𝑃𝑊(𝑥, 𝑦, 𝑐) = ∑ 𝐾 ́(𝐶 ́) 𝐶 𝑐= 1 ⋅ 𝐷𝑊(𝑥, 𝑦, 𝑐)(7)

Where DW denotes the output of the depthwise convolution for channel c, PW is the output of the pointwise convolution for channel c′, 𝐾 𝑐 is the kernel for the depthwise convolution, and K′ is the 1×11×1 kernel for the pointwise convolution. C is the number of channels. Activation functions such as the Swish function, defined as 𝑓(𝑥) = 𝑥 ⋅ 𝑠𝑖𝑔𝑚𝑜𝑖𝑑(𝑥), are applied after convolutional operations to introduce non-linearity, enabling the network to learn complex features. By integrating these elements, EfficientNetB0 provides a powerful and efficient framework.

Dynamic Blurring for Privacy Preservation

we address privacy concerns in video-based monitoring by implementing dynamic blurring for privacy preservation. This process involves the selective obfuscation of regions of interest (ROI) within video frames, specifically targeting identifiable features of individuals to maintain anonymity while preserving the utility of the data for fall detection. The identification of ROIs for blurring is governed by a detection function 𝐷(𝐹 𝑖𝑛 , 𝜃), where 𝐹 𝑖𝑛 represents an input frame, and 𝜃 denotes the parameters of the detection model, which may include facial recognition, pose estimation, or other relevant feature detection algorithms. The output of this function is a set of bounding boxes 𝐵 = {𝑏 1 , 𝑏 2 , . . . , 𝑏 𝑛 }, where each 𝑏 𝑖 specifies the coordinates and dimensions of an ROI within the frame. The dynamic blurring is then applied to these identified ROIs using a Gaussian blur operation, mathematically described as:

𝐺(𝑥, 𝑦, 𝜎) = 1 2𝜋𝜎 2 𝑒 − 𝑥 2 +𝑦 2 2𝜎 2(8)

Where (𝑥, 𝑦) are the coordinates relative to the center of the kernel, and 𝜎 is the standard deviation, which controls the extent of blurring. The size of the kernel, 𝑘 × 𝑘, is chosen based on the desired level of blurriness, typically set to several times the value of 𝜎 to ensure that the edges of the kernel contribute negligibly to the blur. The application of the Gaussian blur to an ROI 𝑏 𝑖 within the frame 𝐹 𝑖𝑛 can be represented as:

𝐹 𝑏𝑙𝑢𝑟𝑟𝑒𝑑 (𝑥, 𝑦) = (𝐹 𝑖𝑛 * 𝐺)(𝑥, 𝑦) = ∑ ∑ 𝐹 𝑖𝑛 𝑏 𝑛= −𝑏 𝑎 𝑚= −𝑎 (𝑥 − 𝑚, 𝑦 − 𝑛) ⋅ 𝐺(𝑚, 𝑛, 𝜎) (9)

for all (𝑥, 𝑦) within 𝑏 𝑖 , where * denotes the convolution operation, and a and b are half the width and height of the Gaussian kernel, respectively.

Temporal Analysis with LSTM

The LSTM network is a specialized RNN designed to model temporal dependencies in sequence data effectively. Its architecture is uniquely suited to address the vanishing gradient problems, enabling it to capture long-term dependencies. An LSTM unit comprises three main gates: the input gate (i), the forget gate (f), and the output gate (o), each responsible for regulating the flow of information. We utilized bidirectional LSTM with the following structure, as shown in Figure 2.

Classification Framework

Following the extraction of temporal features, the next step involves classification, which is classified between fall and non-fall events. This process typically involves passing the LSTM output through one or more fully connected layers in a SoftMax layer for binary classification:

ℎ 𝑧 = 𝑊 ℎ ⋅ ℎ 𝑡 + 𝑏 ℎ (10) 𝑝 = 𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑧) = 𝑒 𝑧 𝑖 ∑ 𝑒

𝑧 𝑗 𝑗 (11) where ℎ 𝑡 is the output from the LSTM at time t, 𝑊 ℎ and 𝑏 ℎ are the weights and biases for the dense layer, respectively, z is the logit, and p represents the probabilities for each class obtained through the SoftMax function. The class with the highest probability is selected as the predicted class for each input sequence. This framework facilitates the effective classification of video sequences into fall or non-fall categories based on the temporal patterns identified by the LSTM network.

Training Process

The training of the integrated model is underpinned by a mathematical framework that includes the definition of a loss function, the selection of an optimization algorithm, and the application of regularization techniques to prevent overfitting. The loss function L quantifies the discrepancy between the predicted outputs p and the true labels y. For binary classification tasks, such as fall detection, the binary cross-entropy loss is commonly used:

𝐿(𝑦, 𝑝) 1 𝑁 ∑ [𝑦 𝑖 log(𝑝 𝑖 ) + (1 − 𝑦 𝑖 ) log(1 − 𝑝 𝑖 )] 𝑁 𝑖=1 (15)

Where N is the number of samples, 𝑦 𝑖 is the true label, and 𝑝 𝑖 is the predicted probability for the i-th sample. The optimization of the model parameters is achieved through Stochastic Gradient Descent (SGD), which iteratively updates the weights W based on the gradients of the loss function:

𝑊 𝑡+1 = 𝑊 𝑡 − 𝜂𝛻𝐿 (16)

Where 𝜂 is the learning rate, and ∇L denotes the gradient of the loss function for the weights at time t. L2 regularization and dropout techniques are applied to mitigate overfitting by adding a penalty term to the loss function or randomly omitting units from the network during training, respectively. The backpropagation process facilitates the computation of gradients ∇L through the network, employing the chain rule to propagate errors from the output layer back through the LSTM and EfficientNetB0 layers, enabling the model to learn and adjust its parameters to minimize the loss function. predictions, along with false positive (FP) and false negative (FN) predictions. In this case, the matrix reveals perfect classification on the test data, with all fall events being correctly identified (6 TP) and no ADL events being misclassified as falls (0 FN), implying an exceptional level of model performance.

Results and Analysis

Figure 4. Confusion metric of unseen test videos

The Receiver Operating Characteristic (ROC) curve in Figure 5. plots the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. The area under the curve (AUC) in this ROC curve approaches 1, which suggests excellent model performance, with a high true positive rate and a low false positive rate across threshold values.

Figure 5. ROC curve of the proposed model on unseen test data

Figure 6 shows the model's performance on real test videos; the model correctly predicted this particular scenario as a 'Fall,' corroborated by the RGB image on the right. This clearly shows an individual in a prone position on the floor. Moreover, the depth image reveals the successful application of the dynamic blurring method. The individual's features are indistinguishable, and the privacy-preserving objective of the method is evident. The contours and the general posture of the person are discernible, which is sufficient for fall detection purposes, but the finer details necessary for personal identification have been effectively obfuscated. The blurring technique implemented in the system is designed to activate upon detecting a human figure within the video frame, applying a Gaussian blur where the person is detected. This ensures that any potentially sensitive information is rendered non-identifiable, addressing privacy concerns paramount in real-world applications of surveillance-based systems. The obscured depth image confirms that the privacy-preserving measures do not impede the algorithm's ability to detect a fall. As shown in Table 1, the comparative analysis of fall detection methodologies yields a substantive understanding of the advancements and varying efficacies of diverse approaches in this research domain. The table encapsulates the True Positive Rate (TPR), True Negative Rate (TNR), and overall Accuracy, serving as pivotal metrics for the assessment of each method. The model presented by Eltahir et al. [40] manifests a commendable balance between sensitivity and specificity, with a TPR of 95.88% and a TNR of 97.02%, culminating in an accuracy of 97.56%. Chan Su's model slightly improves sensitivity at 98.07% and specificity at 99.03%, with an analogous accuracy of 98.06%. These two models set a robust baseline in fall detection, evidencing high efficacy. The single-stream models using RGB and Optical Flow (OF) data individually attain a TPR of 100%, indicative of their flawless identification of fall events. However, their specificity scores, 96.61% and 96.34%, respectively, although high, suggest a slightly less robust capacity to classify non-fall activities accurately. This slight discrepancy is reflected in their accuracy scores, which, while impressive at 96.99% and 96.75%, do not reach the pinnacle of Chan Su's model. The multi-stream approach amalgamating RGB, OF, and Pose Estimation (PE) data represents a significant leap forward, yielding a perfect TPR and an enhanced TNR of 98.61%, leading to an accuracy of 98.77%. This approach underscores the utility of integrating multiple data streams for improved specificity without compromising sensitivity. EfficientNet-B0, despite a lower TPR of 93.33%, achieves a perfect TNR of 100%. This accentuates the model's exceptional performance in identifying non-fall events, though it falls short of the multi-stream model's balanced accuracy. The improved YOLOv5s model and the single frame human binary image approach using YOLOv5s do not disclose TPR or TNR but report accuracies of 97.2% and 96.7%, respectively. While these figures suggest competent models, the lack of detailed TPR and TNR data precludes a complete comparative analysis. Our proposed methodology establishes a new benchmark, recording a flawless TPR and TNR of 100% and an unmatched accuracy of 100%. This unprecedented performance indicates a superior ability to correctly identify fall incidents and an unparalleled precision in confirming non-fall activities.

Conclusion

In this research, we have successfully developed and evaluated a novel video-based fall detection system that prioritizes privacy without compromising the real-time detection efficacy of elderly falls. By integrating EfficientNetB0 and LSTM networks, our methodology ensures robust feature extraction and accurate fall event classification. The introduction of dynamic blurring as a privacy-preserving technique represents a significant advancement, allowing for anonymizing identifiable features within video frames while maintaining the system's operational integrity. Our findings reveal that this approach achieves perfect accuracy, recall, precision, and area AUC scores. It also effectively addresses the critical privacy concerns of video surveillance in sensitive environments such as homes and elderly care facilities. Implementing dynamic blurring ensures that the privacy of monitored individuals is safeguarded, setting a new precedent in the ethical application of surveillance technologies in healthcare.

Our future research will focus on further enhancing the adaptability and generalizability of the system across diverse settings and populations. This includes exploring additional privacypreserving mechanisms and integrating multimodal data sources to enrich the system's contextual understanding. This research contributes significantly to elderly care technology, presenting a practical solution to the long-standing challenge of balancing effective fall detection with stringent privacy requirements. Our work advances the technological capabilities in this domain and addresses critical ethical considerations. This paves the way for broader acceptance and deployment of video-based monitoring systems in healthcare settings.