A Unified Framework for Human Activity Recognition Data Preprocessing

Abstract

Human Activity Recognition (HAR) is widely applied in smart homes, healthcare, and ambient intelligence to monitor human behavior, particularly for the Activities of Daily Living (ADL) in elderly care. The heterogeneity of sensor configurations and monitored activities presents challenges in reusing datasets from one environment or person in other settings. This discussion paper describes a hybrid framework for HAR that integrates both data-driven and knowledge-driven approaches. Core concepts are represented in an abstract model that maps raw sensor activations into structured representations using Functional Areas, Detector Units, and Structural Human Activities. Deep learning models are then applied to the standardized data to enhance generalization across diferent environments.

Keywords

• eol>Activities of Daily Living (ADL)
• Human Activity Recognition (HAR)
• Gated Recurrent Unit (GRU)
• Data preprocessing
• Data integration
• Deep learning for HAR

1. Introduction

eficient and efective way to model temporal dependencies in sequential sensor data while maintaining lower computational complexity.

Sensor data are mapped into Functional Areas, which generalize activity zones, and Detector Units, which abstract sensor types, facilitating cross-environment generalization. The GRU architecture is ifne-tuned and evaluated through two complementary training strategies: in the Holistic Approach, a single GRU-based model is trained on multiple datasets; in the Reductionist Approach, ensemble learning via bagging is applied, where the GRU-based model is trained on diferent subsets of the datasets, and their results are combined to make a final prediction.

The paper is structured as follows: Section 2 reviews existing HAR methodologies, Section 3 describes our unified HAR model, Section 4 introduces the used datasets and challenges in merging diferent datasets, Section 5 describes the neural network architecture and the training strategies, Section 6 presents experimental results, and Section 7 concludes the study.

2. Related Work

Research in HAR can be categorized into data-driven, knowledge-driven, and hybrid approaches.

Data-Driven Approaches. Existing methodologies in activity recognition focus on overcoming dataset limitations and improving generalization across domains. Chiang et al. [ 9 ] introduced a featurebased knowledge transfer framework that uses transfer learning to handle discrepancies between training and testing datasets, achieving 8% higher accuracy and reducing the need for labeling the target domain. Feuz and Cook [ 10 ] proposed Feature Space Remapping (FSR), a heterogeneous transfer learning technique that aligns features from source and target domains for activity classification. Azkune et al. [ 11 ] developed two data-driven systems, Seminar-U and Seminar-S, to handle labeled and unlabeled activities in the source domain. Myagmar et al. [ 12 ] presented Heterogeneous Daily Living Activity Learning (HDLAL), which creates a domain-invariant feature space and employs ensemble classification for multi-label activity recognition. Lastly, Sanabria et al. [ 13 ] combined Bi-GAN (Bidirectional Generative Adversarial Networks) and KMM (Kernel Mean Matching) for unsupervised domain adaptation, facilitating feature transfer between heterogeneous domains in daily activity recognition.

Knowledge-Driven Approaches. Knowledge-driven approaches in activity recognition leverage ontologies and semantic reasoning to enhance event interpretation. Marjan et al. [ 14 ] proposed Ecare@home, integrating smart home data into an ontology for semantic event interpretation and activity recognition using an incremental answer set solver. Wemlinger and Holder’s SCEAR [ 15 ] also uses a common ontology and reasoning to recognize activities. Ye et al. introduced Slearn [ 16 ], an ensemble learning approach that utilizes semantic mapping for knowledge transfer across datasets, later developing XLearn [ 17 ], which employs ontologies to map sensor data to daily activities. XLearn utilizes clustering and ensemble learning to identify and classify activities from sparse labeled data. Ye et al. also proposed a knowledge model [ 18 ] for eficient feature space remapping and uncertainty inference, improving classification accuracy.

Alternative Approaches. D. Cook [ 19 ] proposed a method for learning generalized activity models that abstract from specific environments using supervised and semi-supervised machine learning. By applying classifiers like naïve Bayes, hidden Markov models, and conditional random fields on datasets from the CASAS Smart Home project, the study showed that ensemble classifiers with semi-supervised learning improved performance. In another study [ 20 ], Cook classified transfer learning techniques for cross-environment human activity recognition (HAR) into template matching, generative, and discriminative approaches. Challenges in transferring knowledge across diferent domains, datasets, and sensor modalities were addressed, and various transfer learning types were introduced, such as informed/uninformed supervised/unsupervised approaches. Masciadri et al. [ 21 ] proposed a knowledgedriven system for recognizing Activities of Daily Living using unobtrusive environmental sensors. The architecture combines unsupervised temporal segmentation with semantic reasoning and includes a resident-adaptive layer that personalizes the underlying knowledge base to improve recognition accuracy. Lastly, Yu et al. [ 22 ] proposed a sensor mapping approach for heterogeneous smart homes using algorithms to identify the most similar source homes and optimize sensor mapping. Deep Adversarial Transfer Network (DANN) was then applied for HAR, facilitating cross-environment generalization.

A diferent perspective is adopted in the works surveyed by Bertrand et al. [ 23 ], which explore the use of process mining techniques in smart spaces. For instance, Leotta et al. [ 24 ] apply process mining to derive interpretable models of human habits from raw sensor data. In contrast, our approach ofers a complementary strategy by introducing an abstraction layer that standardizes and generalizes behavior modeling across heterogeneous datasets, thereby enhancing the available data for application in data-driven approaches across new scenarios (e.g., new flats, diverse sensors).

Despite these advancements, achieving a unified HAR framework remains a challenge. This study addresses this gap by introducing a standardized abstraction layer that generalizes across diferent sensor configurations and home environments.

3. HAR Model

To allow HAR dataset integration and improve model generalization, we introduce a unified HAR model, whose core components include: 1. 1. Physical Environment Layer: Represents real-world entities such as rooms and sensors. • 1. Room (R): A physical space where activities occur (e.g., Kitchen, Bathroom, Bedroom). • 2. Sensor (S): A hardware component detecting an activity (e.g., Motion Sensor, Power

Sensor). 2. 2. Functional Representation Layer: Introduces Functional Areas and Detector Units to abstract the physical environment.

• 1. Functional Area (FA): Abstracts multiple rooms (or parts of the rooms) based on their primary function (e.g., Sleeping Area, Eating Area). • 2. Functional Unit (FU): Represents interactive objects (e.g., Bed, Sink, Refrigerator). • 3. Detector Unit (DU): Aggregates multiple sensor readings for a specific function (e.g., PIR

Presence Sensor and Door Sensor to detect presence in a functional area). 3. 3. Activity Recognition Layer: Defines Structural Human Activities based on sensor interactions.

We selected a subset of the Katz’s Basic Activities of Daily Living [ 25 ] meaningful for elderly care through sensor monitoring and added some activities commonly found in many datasets like Sleep, Relax and Laundry. The set of key activities we selected includes: • 1. Sleep (e.g., Night sleep, Napping) • 2. Eat (e.g., Cooking, Eating meals) • 3. Relax (e.g., Watching TV, Reading) • 4. Personal hygiene (e.g., showering, washing hands, brushing teeth) • 5. Toileting (e.g., Bathroom visits) • 6. Laundry (e.g., using washing machine)

For example, Figure 1 represents the Sleeping activity: this may occur in the bedrooms or, in the case of a nap, in the living room. These places are denoted as a single functional area for sleeping, associated with distinct Functional Units - the beds, the sofa and the television - which are monitored by diferent sensors in the various rooms to provide data about activity patterns. Sensors are abstracted into generic detector units.

4. Datasets and Integration Challenges

The main datasets employed for Human Activity Recognition (HAR) based on ADLs are summarized in Table 1.

The variability in the datasets can contribute to the development of robust activity recognition systems for smart homes, making the system more adaptable to diferent home environments and individuals. On the other side, the integration of the diferent sensor-based datasets presents several challenges. Besides mapping the physical environments into Functional Areas and abstracting raw sensor readings into Detector Units, we needed to align sensor activations to fixed time intervals. We opted to consider a granularity of 5 seconds. This choice was based on the predictable temporal patterns of activities of daily living (ADLs), as well as factors like room size and elderly individuals’ walking speed. The 5-second interval captures movements efectively without excessive granularity.

Given that most datasets did not conform to the 5-second interval structure, data imputation was applied to adjust sensor data. A forward fill method was used, with careful attention to avoid filling extended time gaps. Moreover, class imbalance was a major issue, as certain activities (e.g., sleeping) were much more prevalent than others. An "Other" class was introduced to handle infrequent or x x x x x h c n u L t a E x x x x x r e n n i D t a E x x x x x x p e e l S p a N x a l e

R x x x x x x x x x x x x x g n i p e e l S e c n e s e r P x x x x x a l feRO / n O x x x a l e R r e w o P x x x x x x a l e R e c n e s e r P x x x x x t e li foTO / n O unclassified activities and ensure temporal continuity.

Based on the selected datasets and the activities deemed relevant for our application scenarios, Tables 2 and 3 illustrate the coverage of these activities and the presence of corresponding Detector Units within each dataset, respectively.

To design the input for our network, we selected a fixed-length representation consistent with the original datasets, which employ binary integer arrays. We retained this encoding to maintain compatibility and ensure comparability. As illustrated in Figure 2, the array comprises 15 elements representing Detector Units (DUs), arranged such that units within the same Functional Area (FA) are placed adjacently to preserve spatial relationships. A final element, corresponding to the hour of the day and normalized prior to training, is appended to complete the input vector. In the example, two detector units associated with the sleeping functional activity are active at 5 am.

5. Neural Network Architecture and Training Approaches

The Gated Recurrent Unit (GRU) was selected for its eficiency in handling sequence data and capturing long-term dependencies. We employed GridSearchCV from the scikit-learn library to fine-tune the GRU model. Key configurations identified during fine-tuning included the model’s input shape, batch normalization for stability, GRU layers with 128, 64, and 32 units, followed by two dense layers with swish activation, and a softmax output layer.

We explored two distinct methodologies for model training: the Holistic Approach and the Reductionist Approach.

In the Holistic Approach, a single GRU model was trained on a unified dataset combining all the datasets. The dataset was split into a training set and a testing set (30% for testing). A 3-fold crossvalidation was applied to reduce errors arising from diferent validation sets. The goal was to create a model capable of generalizing across diferent datasets, accommodating variations in sensor configurations and house planimetry.

In contrast, the Reductionist Approach leverages ensemble methods, specifically Bagging, to improve model performance. Bagging combines the predictions of multiple independent models, referred to as weak learners. Each GRU model was trained on a distinct subset of data from individual datasets, and their predictions were aggregated using a majority voting mechanism. This approach addressed class imbalance and model robustness, following the arguments by Ferrari et al. [ 29 ] on combining personalization with generalizable deep learning frameworks for HAR.

6. Experimental Results

We trained the model using the following datasets: the whole Van Kasteren dataset, OrdonezA and OrdonezB, CasasHH101, Tapia1 and Tapia2. Here, only the main results are reported.

The GRU model was trained on individual datasets and tested on others. Performance metrics showed high loss and low evaluation scores, demonstrating the challenge of generalizing models trained on diverse sensor configurations and data distributions. We obtained an accuracy of 0.55 and F1 score of 0.33.

The Holistic Approach, using a single model trained on the entire dataset, achieved competitive performance with an F1 score of 0.72 and accuracy 0.73.

The model’s performance varied when trained on diferent datasets. Training on a rich dataset, including Tapia2, resulted in better performance, while excluding it led to a decline in metrics.

7. Conclusions and Future Work

This study introduced a unified framework for integrating multiple datasets in Human Activity Recognition (HAR), with a focus on monitoring Activities of Daily Living (ADL) in home environments. By mapping rooms and sensors into a standardized abstraction layer, we mitigated dataset-specific variations and enhanced cross-environment generalization. The proposed approach demonstrated strong performance in recognizing key activities, in particular, with the holistic model. However, challenges remain in improving model adaptability to new environments and sensor configurations. Future work will extend the dataset pool to refine the abstraction process, exploring alternative machine learning architectures for enhanced accuracy, such as Transformer-based models. Moreover, the holistic model will be further validated in real-world deployments to assess its robustness and adaptability.

Declaration on Generative AI

During the preparation of this work, the author(s) used ChatGPT-4 and Grammarly for suggestions on grammar and spelling and to improve writing style. After using these tools/services, the authors reviewed and edited the content as needed and took full responsibility for the publication’s content.

References

[1] United Nations Department of Economic and Social Afairs, Growing at a slower pace: World population expected to reach 9.7 billion in 2050 and could peak at nearly 11 billion around 2100, 2024 . URL: https://www.un.org/en/desa/ growing-slower -pace-world-population-expected- reach- 97 -billion- 2050 - and - could , accessed: 2025 -02-25.

[2] F. Demrozi , G. Pravadelli , A. Bihorac , P. Rashidi , Human activity recognition using inertial, physiological and environmental sensors: A comprehensive survey , IEEE Access 8 ( 2020 ) 210816 - 210836 . doi: 10 .1109/ACCESS. 2020 . 3037715 .

[3] D. Bouchabou , S. M. Nguyen , C. Lohr , B. LeDuc, I. Kanellos, A survey of human activity recognition in smart homes based on iot sensors algorithms: Taxonomies, challenges, and opportunities with deep learning , Sensors 21 ( 2021 ). URL: https://www.mdpi.com/1424-8220/21/18/6037. doi: 10 . 3390/s21186037.

[4] F. Veronese , A. Masciadri , S. Comai , M. Matteucci , F. Salice , Behavior drift detection based on anomalies identification in home living quantitative indicators , Technologies 6 ( 2018 ) 16 . URL: http://dx.doi.org/10.3390/technologies6010016. doi: 10 .3390/technologies6010016.

[5] D. Hooda , R. Rani , Ontology driven human activity recognition in heterogeneous sensor measurements , Journal of Ambient Intelligence and Humanized Computing 11 ( 2020 ) 5947 - 5960 . URL: http://dx.doi.org/10.1007/s12652-020 -01835-0 . doi: 10 .1007/s12652-020-01835-0.

[6] X. Huang , S. Zhang, Human activity recognition based on transformer in smart home , in: Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning , CACML '23 , Association for Computing Machinery, New York, NY, USA, 2023 , p. 520 - 525 . URL: https://doi.org/10.1145/3590003.3590100. doi: 10 .1145/3590003.3590100.

[7] F. Salice , A. Masciadri , G. Di Blasio , M. Venturelli , S. Comai , Adlnet: A 1d-cnn for activity of daily living recognition in smart homes , in: J. Bravo , G. Urzáiz (Eds.), Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023 ), Springer Nature Switzerland, Cham, 2023 , pp. 76 - 87 . URL: https://doi.org/10.1007/978-3- 031 -48642- 5 _8. doi: 10 .1007/978-3- 031 -48642-5\_8.

[8] D. J. Cook , A. S. Crandall , B. L. Thomas , N. C. Krishnan , Casas: A smart home in a box , Computer 46 ( 2013 ) 62 - 69 . doi: 10 .1109/ MC . 2012 . 328 .

[9] Y.-T. Chiang , C.-H. Lu , J. Y. -J. Hsu , A feature-based knowledge transfer framework for crossenvironment activity recognition toward smart home applications , IEEE Transactions on HumanMachine Systems 47 ( 2017 ) 310 - 322 . doi: 10 .1109/THMS. 2016 . 2641679 .

[10] K. D. Feuz , D. J. Cook , Transfer learning across feature-rich heterogeneous feature spaces via feature-space remapping (fsr) , ACM Transactions on Intelligent Systems and Technology 6 ( 2015 ) 1 - 27 . URL: http://dx.doi.org/10.1145/2629528. doi: 10 .1145/2629528.

[11] G. Azkune , A. Almeida , E. Agirre, Cross-environment activity recognition using word embeddings for sensor and activity representation , Neurocomputing 418 ( 2020 ) 280 - 290 . URL: https://www. sciencedirect.com/science/article/pii/S0925231220313230. doi:https://doi.org/10.1016/j. neucom. 2020 . 08 .044.

[12] B. Myagmar , J. Li , S. Kimura , Heterogeneous daily living activity learning through domain invariant feature subspace , IEEE Transactions on Big Data 7 ( 2021 ) 922 - 929 . doi: 10 .1109/TBDATA. 2020 . 2977626 .

[13] A. R. Sanabria , F. Zambonelli , J. Ye , Unsupervised domain adaptation in activity recognition: A gan-based approach , IEEE Access 9 ( 2021 ) 19421 - 19438 . doi: 10 .1109/ACCESS. 2021 . 3053704 .

[14] M. Alirezaie , J. Renoux , U. Köckemann , A. Kristofersson , L. Karlsson , E. Blomqvist , N. Tsiftes , T. Voigt , A. Loutfi , An ontology-based context-aware system for smart homes: E-care@home , Sensors 17 ( 2017 ). URL: https://www.mdpi.com/1424-8220/17/7/1586. doi: 10 .3390/s17071586.

[15] Z. E. Wemlinger , L. B. Holder , Cross-environment activity recognition using a shared semantic vocabulary , Pervasive and Mobile Computing 51 ( 2018 ) 150 - 159 . URL: https://www.sciencedirect. com/science/article/pii/S1574119218302621. doi:https://doi.org/10.1016/j.pmcj. 2018 . 10 .004.

[16] J. Ye , Slearn: Shared learning human activity labels across multiple datasets ( 2018 ) 1 - 10 . doi: 10 . 1109/PERCOM. 2018 . 8444594 .

[17] J. Ye , S. Dobson , F. Zambonelli , Xlearn: Learning activity labels across heterogeneous datasets , ACM Trans. Intell. Syst. Technol . 11 ( 2020 ). URL: https://doi.org/10.1145/3368272. doi: 10 .1145/ 3368272.

[18] J. Ye , Shared learning activity labels across heterogeneous datasets , Journal of Ambient Intelligence and Smart Environments 13 ( 2021 ) 77 - 94 . URL: http://dx.doi.org/10.3233/AIS-210590. doi: 10 . 3233/ais-210590.

[19] D. Cook , Learning setting-generalized activity models for smart spaces , IEEE Intelligent Systems 27 ( 2012 ) 32 - 38 . doi: 10 .1109/MIS. 2010 . 112 .

[20] D. J. Cook , Transfer learning for activity recognition: A survey , Knowl Inf Syst 36 , 537 - 556 ( 2013 ).

[21] A. Masciadri , S. Comai , F. Salice , Smare: Semi-supervised method for activities of daily living recognition , in: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019 , pp. 3403 - 3409 . doi: 10 .1109/SMC. 2019 . 8914279 .

[22] Y. Yu , Z. Hao , G. Li , Y. Liu , R. Yang , H. Liu, Optimal search mapping among sensors in heterogeneous smart homes , Mathematical Biosciences and Engineering 20 ( 2022 ) 1960 - 1980 . URL: http://dx.doi. org/10.3934/mbe.2023090. doi: 10 .3934/mbe.2023090.

[23] Y. Bertrand , B. Van den Abbeele, S. Veneruso, F. Leotta , M. Mecella , E. Serral , A survey on the application of process mining to smart spaces data , in: M. Montali , A. Senderovich , M. Weidlich (Eds.), Process Mining Workshops , Springer Nature Switzerland, Cham, 2023 , pp. 57 - 70 .

[24] F. Leotta , M. Mecella , J. Mendling , Applying process mining to smart spaces: Perspectives and research challenges , in: A. Persson , J. Stirna (Eds.), Advanced Information Systems Engineering Workshops , Springer International Publishing, Cham, 2015 , pp. 298 - 304 .

[25] S. Katz , Assessing self-maintenance: Activities of daily living, mobility, and instrumental activities of daily living , Journal of the American Geriatrics Society 31 ( 1983 ) 721 - 727 . URL: http://dx.doi. org/10.1111/j.1532- 5415 . 1983 .tb03391.x. doi: 10 .1111/j.1532- 5415 . 1983 .tb03391.x.

[26] T. van Kasteren , A. Noulas , G. Englebienne , B. Kröse , Accurate activity recognition in a home setting , in: Proceedings of the 10th International Conference on Ubiquitous Computing , UbiComp '08, Association for Computing Machinery, New York, NY, USA, 2008 , p. 1 - 9 . URL: https://doi.org/ 10.1145/1409635.1409637. doi: 10 .1145/1409635.1409637.

[27] E. M. Tapia , S. S. Intille , K. Larson , Activity Recognition in the Home Using Simple and Ubiquitous Sensors, Springer Berlin Heidelberg, Berlin, 2004 , p. 158 - 175 . URL: http://dx.doi.org/10.1007/ 978-3- 540 -24646-6_ 10 . doi: 10 .1007/978-3- 540 -24646-6\_ 10 .

[28] F. J. Ordóñez , P. De Toledo , A. Sanchis , Activity recognition using hybrid generative/discriminative models on home environments using binary sensors , Sensors 13 ( 2013 ) 5460 - 5477 . URL: https: //www.mdpi.com/1424-8220/13/5/5460. doi: 10 .3390/s130505460.

[29] A. Ferrari , D. Micucci , M. Mobilio , P. Napoletano , Deep learning and model personalization in sensor-based human activity recognition , Journal of Reliable Intelligent Envi-