discovery. Models usually employed for Ambient Intelligence (AmI) in smart homes are usually obtained directly from sensor logs composed by timestamped sequences of sensor measurements. Such approaches, still efective at diferent tasks, have the drawback of producing representations dificult to read and validate. In this paper we propose a tool, called Visual Process Maps (VPM), intended to allow the analysis of human routines at the human action level thanks to log preprocessing and the application of process Proceedings of the Demonstration & Resources Track, Best BPM Dissertation Award, and Doctoral Consortium at BPM
The term smart space refers to an environment enriched with a set of devices (e.g., sensors and
actuators) which aims at providing intelligent services to the human user, realizing the paradigm
known as ambient intelligence (AmI) [
The literature in the area proposes several solutions to analyze sensor logs and to automatically
perform actions on the basis of the current context, user preferences and habits. All of these
approaches are based on models which describe the relationship between (i) the behavior of
a smart space’s inhabitant(s), (ii) the specific contextual state of the environment, and (iii) a
portion of sensor measurements from the log. Most of these techniques anyway, and especially
those based on machine learning techniques (the so called learning-based methods), only rely
on raw sensor measurements, whereas human actions are usually neglected, making it dificult
to visually inspect daily routines and habits [
Process Mining [
Therefore, applying techniques from the BPM area can be a good compromise to overcome the gap between raw sensor measurements and human actions. Due to the BPM’s input limitation, a log-preprocessing step is required to infer human actions from the sensor log.
In this paper we propose the Visual Process Maps (VPM) system, consisting of a complete pipeline formed by (i) a tool for the visual analysis of sensor logs, (ii) a method to transform raw movement measurements into actions, and (iii) a method to identify and visually analyze precedence relationships between human actions. The tool and related resource can be downloaded from https://www.diag.uniroma1.it/leotta/demos/vpm_bpm2021.html.
VPM is a tool that allows to graphically represent log activations and then to extract habit models out of them. The intended user of such a tool is a domain expert, who analyses the activities in the environment (e.g., an house) and takes decisions about the design of the space itself.
The literature about representing models of human habits is wide. In this section, we will highlight the diferences between our work and those representation tools available in literature, which produce a human readable output that is easy to validate (once the representation is known).
In [
A survey about context modeling and reasoning techniques over contexts and situations is
provided in [
Recently, the BPM research community has applied process mining techniques to smart
spaces [
These first attempts to apply techniques taken from the business process management were
focused on workflow specifications to anticipate user actions. A workflow is composed by a set of
tasks related by qualitative and/or quantitative time relationships. In [
This category of workflow-based modeling techniques (like SPUBS) can, in the case of highly variable processes, produce unreadable models; furthermore this approach requires the sensor log to be already segmented before mining.
With VPM, indeed, we decided to apply a specific technique named fuzzy mining that extracts models, which are more suitable than workflows employed in SPUBS, to describe human habits that are flexible by nature. As an advantage, this representation naturally supports visual analytics.
Finally, we want to mention Sitivus, a tool adopting visual analytics in smart spaces [
Sensor Log
Trajectory Visualization Tool
Habit Extraction
Extracted Models their work, they remark the importance of providing domain experts with a tool allowing to graphically inspect models of smart spaces. However, with respect to our work, sitivus is focused on modeling and representing situations more than habits. A situation is an interpretation of the context at a higher level and can comprehend the state of multiple users and consider the activities that are executed by each of them (e.g., “users are having lunch”), while a habit is focused on the routine of the user on specific time intervals (e.g., “what the user usually does after work?”).
As discussed in Section 1, in order to apply process mining techniques, the sensor log must
be translated into a suitable event log ℰ. In this work, we adapted a portion of the TRACLUS
algorithm by [
It considers the entire sensor data log as a single complex trajectory in which it can identify several sub-trajectories, each one describing a portion (a sub-log) of the original sensor log, which are related to the same action. This segmentation is based on finding the so called “characteristic points”: points in which the trajectory’s behaviour changes rapidly. Finally, it classifies each sub-trajectory by considering information about its movement and also which are the sensors involved and for how long, and labels them with an action (MOVEMENT, AREA or STAY), and their relative location inside the smart environment (e.g., <STAY Kitchen_table>). Now the log can be visually analysed by a human domain expert, who could be interested in replaying the log (or a portion of it) by employing the trajectory visualization tool (see Figure 2). Furthermore, it can now be processed through process mining techniques.
The event log, obtained from the previous step (Section 3.1), is fed into the fuzzy miner. The fuzzy miner produces a directed graph where each node represents a task (or in our case a human action) and each arc represents a precedence constraint between an action and its successor. Nodes and arcs are weighted, respectively with the number of times an action appears in the log and the number of times that an action follows another in the log.
These information are used to extract frequent sequences of human actions (i.e., paths inside the graph), by using an entire day as time reference.
Finally, we merge all these sequences into single activity models. As this approach was evaluated on the partially labeled Aruba dataset, we had at our disposal a ground truth consisting of sequences labeled with the name of the activity (e.g., Relax, Eating, Sleeping, and others). All frequent sequences extracted were compared with these labeled “ground truth” sequences, by computing the Jaccard coeficient : each sequence is then assigned to the activity label of the ground truth sequence obtaining the maximum score. As a final step, the sequences belonging to the same activity, are combined together: a graph for each activity class, that is also a subgraph of the original graph. The results obtained are very interesting, as the segments extracted are representative of the activity performed.
This paper introduces VPM, a tool for the automated analysis of smart home sensor logs through process discovery.
At the current stage, the proposed graphical representation and, as consequence, models produced by the tool are limited by the type of sensors considered during the analysis. At the current stage, only PIR sensors have been considered; this does not allow to understand precisely the action performed by the user, but only the usage of a device. Supporting other types of sensors, would led to better classify actions and to more readable and accurate models, thus allowing a greater precision of the whole approach. Extending the analysis to other types of sensors will be an important starting point for our future work.
In addition, the employment of the tool in a real scenario would require also automatic segmentation techniques in order to describe human routines at a level of granularity finer than an entire day.