This paper presents the use of a model which mixes logical knowledge and statistical inference to recognize Activities of Daily Living (ADL) from sensors in a smart home. This model called Markov Logic Network (MLN) has different implementations and we propose to compare three of them, from the widely used Alchemy to the new generic framework DeepDive. Finally, we discuss the interest these software products can have for real time activity recognition.
Smart home, as described by De Silva [
The context is a generic term which generally regroup the location of the user,
its current activity, etc. [
Activity recognition has been studied for a while and many techniques have
been used. One of them relies on computer vision which leads to ethical,
acceptance and computational problems. To get around this issue, many projects
use only simple and ubiquitous sensors [
Given all these kinds of data, inference have been done following two main
approaches. On the one hand, a set of rules can be de ned by a domain expert to
infer high-level information from low-level sensors' data [
Main implementations of statistico-relational model will be compared in Section 3 against a multi-modal corpus. Finally, we will discuss the different perspectives considered in Section 4. 2
Markov Logic Network (MLN) is one of the statistico-relational models,
primarily introduced by Domingos et al. [
MLN is used to infer the most probable state of a world given some evidences
which is called Maximum A posteriori Probability/Most-Probable Explanation
(MAP/MPE) inference. This can be easily done using a SAT solver able to
handle the weights of the formulas [
With the logical models, a domain expert is needed to transfer its knowledge. MLNs take a step to avoid this expensive phase, allowing it to learn the weights from an annotated corpus. In the same way, some attempts have been made to learn the structure of a MLN using Inductive Logic Programming techniques.
It exists different implementations of MLN. The most well known is Alchemy3 [
Then, Stanford University launched another project, called Tuffy4, using
a similar implementation to Alchemy moving from C++ to Java and relying
on PostgreSQL. Usage of an external relational database management system
(RDBMS) improves dramatically the time and space efficiency of Tuffy
inference phase [
DeepDive5 is the new project of the Hazy Research group6. It is a generic
framework primarily designed for knowledge base creation (KBC) which can
handle many problems [
Until now, we have measured the learning performance of the different available software. The learning performance is the score of the system when it infers on the exact same data set than the learning one.
In our case, we use a multi-modal corpus which regroups a full range of
sensors values, some computed higher level data (room where the user stayed
the longest, agitation level, etc.) and the actual activity of the user. This corpus is
composed of 26 hours of experiments on 21 persons, divided into 1 minute long
time windows, represented by 94 values [
We ran the following experiments: (1) weight learning and inference with Alchemy; (2) inference with Tuffy (based on Alchemy learning); (3) weight learning and inference with DeepDive. We used Alchemy 1.0, Tuffy-0.3 and DeepDive-0.6.0 on a computer with 24 Intel Xeon CPUs at 2.4GHz with 145GB of RAM running Debian 8 (Jessie) in all of our experiments.
As you can see in Table 1, the execution time of Tuffy and DeepDive is improved by an order of magnitude compared to Alchemy. The small differences in F1-score let's suppose that the three approaches are comparable in performance. We also extract the weights learned by Alchemy and by DeepDive. The correlation between the two sets of weights is about 0.69, highlighting a quite high positive linear correlation which can con rm that both systems work the same way, and that their results can be seen as equivalent.
Learning time
Inferring time
F1-Score Alchemy +3 days 3h 45' 89 % Tuffy 3'51" 90 % DeepDive 23" 10" 91 % Table 1. Execution time and F1-score of different implementations of statisticorelational models
One of the goals of our project is to build a system able to give context information to arti cial intelligence. As explained in the Section 1, the user activity is one of the context components.
To do so, our system must be able to infer user activity in real time (the
user must not feel a lag because of computation time) and to adapt itself to
the user, all along its life. Sensor values are coming all along the day, and so
our system must be able to handle on-line classi cation or incremental learning.
DeepDive is designed as an incremental framework, and can execute the next
learning and inferring phases in less time than the one measured in a one shot
test (see Table 1) as shown in its description paper [
Then, we will try to implement a reinforcement learning algorithm, decreasing weight of irrelevant formulas. This raises different questions, as in which proportion can we change the weight? Can we change only one weight, or does the modi cation must impact other weights? How to know which weights to modify?
Finally, we would like the user be allowed to add or remove some formulas without any technical skills. Once these formulas are added to our model, how can we take them into account? Does our model must be fully re-learned?