=Paper=
{{Paper
|id=None
|storemode=property
|title=Semi-supervised Learning: Predicting Activities in Android Environment
|pdfUrl=https://ceur-ws.org/Vol-960/paper8.pdf
|volume=Vol-960
}}
==Semi-supervised Learning: Predicting Activities in Android Environment==
https://ceur-ws.org/Vol-960/paper8.pdf
Semi-supervised learning: predicting activities in Android
environment
Alexandre Lopes1, João Mendes-Moreira2,3, João Gama3,4
Abstract.1 Predicting activities from data gathered with sensors applications of machine learning and statistics, sample size tends to
gained importance over the years with the objective of getting a be the dominant limitation. The problem of working with data
better understanding of the human body. The purpose of this paper streams is the arrival rate of the examples. When new examples
is to show that predicting activities on an Android phone is arrive at a higher rate than they can be mined, the quantity of
possible. We take into consideration different classifiers, their
unused data grows without bounds as time progresses.
accuracy using different approaches (hierarchical and one step
classification) and limitations of the mobile itself like battery and By building a new Smartphone application we attempt to solve
memory usage. A semi-supervised learning approach is taken in problems consistent with previous undertakings, such as: accuracy,
order to compare its results against supervised learning. The cost, performance among others. We explore matters like: (1) the
objective is to discover if the application can be adapted to the user impact of the app on the phone’s battery lifetime; (2) how long
providing a better solution for this problem. The activities should the interval to collect samples be in order to guarantee
predicted are the most usual in everyday life: walking, running, accurate classifications; (3) the time to create a model; and (4) the
standing idle and sitting. An android prototype, embedding the memory space needed.
software MOA, was developed to experimentally evaluate the ideas All software used is open-source so the experiments can be
proposed here. continued and the application can be improved.
The aim of this work will be to create an application that adapt
1 INTRODUCTION to each new user along time, learning his behavior and becoming
more accurate.
Recognizing human activities with sensors next to the body has
become more important over the years, aiming to create or improve
systems in elder care support, health/fitness monitoring, and 2 RELATED WORK
assisting those with cognitive disorders. Activity recognition is not new. Bao & Intille [1] created a system
It is important to have systems that are practical for the user and capable of recognizing twenty activities with bi-axial
that have the possibility to always be with them whilst not feeling accelerometers positioned in five different locations of the user’s
strange or uncomfortable. Taking this into account we will attempt person. This work led to an important discovery, which was
to use only one sensor instead of a, less practical but more possible to get accurate results predicting activities just using
accurate, system of distributed multi-sensors. acceleration values gathered by a sensor placed on the thigh or
The new generation of smart phones has incorporated many dominant wrist. Despite this work uses twenty activities the most
powerful sensors, such as acceleration sensors (i.e. common activities used in other works [2,9,17] are walking,
accelerometers), GPS sensors etc. They give the opportunity to running, sitting, standing, up and downstairs.
create a system that can always be next to the user and work in Some research exists aiming to create a universal model that can be
real-time. In this work we will focus on the motion sensor of the applied to any user. The idea is to use it in an Android application
cell phone, accelerometer, in order to predict the activity that the in order to measure the physical exercise of the user by predicting
user is performing, as was attempted previously by Bao & Intille his activities [2]. This study uses three classification algorithms
[1]. from WEKA (decision trees J48, logistic regression and multilayer
This problem will be treated as a classification problem using neural networks) to induce models to predict user activities. Other
techniques of semi-supervised learning. This will be done in order studies, that also use the WEKA toolkit, implement common
to take advantage of existing examples (typically unlabeled) from algorithms like Naïve Bayes, decision tables, K-nearest neighbors
the current user. and SVM .
Knowledge discovery systems are constrained by three main The common activities that research tries to predict are walking,
limited resources: time, memory and sample size. In traditional running, sitting, standing, up and downstairs.
1
Gu et al. [3] tried to solve the activity recognition problem with
Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias,
techniques of semi-supervised learning using a large amount of
s/n 4200-465 Porto – Portugal, email: alexolopes89@gmail.com
2
Departamento de Engenharia Informática, Faculdade de Engenharia, unlabeled data, together with the labeled data, to build better
Universidade do Porto, Rua Dr. Roberto Frias, s/n 4200-465 Porto – classifiers. Because semi-supervised learning requires less human
Portugal, email: jmoreira@fe.up.pt effort and gives higher accuracy, it is of great interest both in
3
LIAAD – INESC TEC, Rua de Ceuta, 118, 6º; 4050-190 Porto – Portugal theory and in practice [4].
4
Faculdade de Economia, Universidade do Porto, Rua Dr. Roberto Frias,
s/n 4200-464 Porto - Portugal, email: jgama@fep.up.pt
38
One of the most important aspects of the research, in this field, of major importance since we are dealing with streams in a mobile
is the classifiers’ accuracy and the difficulty of label new instances. context.
Both Masud et al. [15] and Guan et al. [16] use ensemble methods The Naive Bayes algorithm is a classification algorithm based
to increase accuracy in partially labeled data (semi-supervised on Bayes rule and can often outperform more sophisticated
problems). A common thing in all the works is how they try to find classification methods. The Naive Bayes algorithm is based on
the more accurate model, testing multiple classifiers with the same conditional probabilities; it calculates a probability by counting the
data. Authors like Kwapisz et al. [2] showed, when trying to solve frequency of values and combinations of values in the historical
this classification problem using decision trees, that the most data. Bayes' Theorem finds the probability of an event occurring
important attribute to differentiate the activities is the acceleration given the probability of another event that has already occurred. It
they induce on the accelerometer. Domingos et al. [11] showed that assumes that the attributes X1…Xn are all conditionally
decision trees like C4.5 could be outperformed by Hoeffding trees, independent of one another, given the target variable Y. The value
and demonstrated their importance when dealing with streams and of this assumption is that it simplifies dramatically the
limited memory space. The biggest problem of decision trees is representation of P(X|Y), and the problem of estimating it from the
that they assume that all training examples can be stored training data [12]. An important advantage of this algorithm is the
simultaneously in main memory, and are thus severely limited in possibility to calculate the required probabilities in one pass over
the number of examples they can learn from. Still, regarding the the training set. Additionally, it is able to obtain good classification
accuracy, the problem can be solved in a hierarchical way. performance even when trained in a small amount of data. We can
Hierarchical classification splits the initial problem into simpler conclude that this classifier can be trained on an efficient way,
sub-problems. The objective is to have a tree in the end where tests gathering the probabilities of each attribute
are done in each node. The classes contained in different nodes Hoeffding trees [13] operate by collecting, for each leaf node,
from the same level of the tree should be independent [5] so there sufficient statistics of the training instances each leaf contains.
is no possible uncertainty when choosing the path. It is expected to Periodically, these leaves are checked to compare the relative
obtain more accurate classifiers by training them in the split data. merits of each candidate attribute for splitting. The Hoeffding
For activity recognition, this can be done by classifying firstly bound, or similar metric, is used to determine when a candidate is
whether the activity is motion or motionless and, in a second step, better than the others. At this point the leaf is split on the best
classifying it in lying, sitting, standing (if it was classified as attribute, allowing the tree to grow. Typically, information gain is
motionless in the first step) or walking, gentle motion and posture used to rank the merits of the split candidates, although other
translation (if it was classified as motion in the first step). These metrics could be used. In the case of discrete attributes, it is
experiments came to the conclusion that rule-based reasoning can sufficient to collect counts of attribute labels relative to class labels
improve the overall accuracy proving the lustiness of this approach to compute the information gain afforded by a split. There are
[6]. some variations of Hoeffding Trees, based on VFDT (Very Fast
The main drawbacks of using such approaches in a mobile Decision Tree learner) which is a high-performance data mining
phone are the limited battery and memory. Experiments were system [11]. It is effective in taking advantage of massive numbers
carried out to determine how long the data samples provided by the of examples by using a very small constant time per example.
cell accelerometer should be in order to obtain accurate Since we are working with a mobile phone the biggest advantage is
classification. Some experiments were made and it was discovered that Hoeffding trees do not store any examples (or parts thereof) in
that at least they need to be captured for 6s and the interval main memory, requiring only a space proportional to the size of the
between them can be up to 10s [7]. These results are used in our tree and the associated sufficient statistics [11].
experiments as described in section 4. Another thing that has The novelty of our work is the creation of the Android
impact on the cell phone, more specifically in its memory, is how application that records data from the accelerometer. It uses a semi-
the data is saved. Not all the data needs to be saved. Using sliding supervised learning algorithm to process data with a model
windows only the most recent data needs to be available [8]. The previously learned. This model is used to label the unlabeled data
features of the raw accelerometer data that can be retrieved are the in real-time. This new labeled data can be used to train future
mean, the standard deviation, the energy and the correlation [9]. models that fit over the user. In the semi supervised approach we
The usefulness of these features has already been demonstrated [1]. defined a threshold of 70% (value that we assumed to be a good
It allows saving both data and memory. percentage of certainty for a classification) which means that we
In terms of mobile applications, DiaTrace [10] is a system add to the training file the instances classified with 70% or more of
developed to aid in sport activities. The authors do not explain how certainty. We can also define the number of these new instances
they carry out the classification. However they guarantee 95% of that we need to gather in order to create a new model. The older
accuracy if the mobile phone is used in the trousers front pocket. instances are deleted in order to maintain the size of the file.
This is an example of how the market demands this type of
applications.
4 AN ANDROID PROTOTYPE
We have implemented an Android application that records data
3 METHODS
from the accelerometer. We use: (1) sequence-based sliding
The tests were made on Naïve Bayes and Hoeffding Trees [11]. windows [8] in order to save memory; and (2) the method of duty
These two algorithms were chosen because some studies showed cycles [7] in order to save battery.
that Naïve Bayes can predict equally as well as decision trees In sequence-based sliding windows an amount of data is
(Langley, Iba, & Thomas 1992; Kononenko 1990; Pazzani 1996) defined. The file will have only the amount of data that the
and Hoeffding trees can learn in a very small constant time what is sequence-based sliding window allows. If new data is added it
39
replaces the oldest data in order to keep the size stipulated by the has two classifications: (1) The first one classifies the data into
window. Dynamic or Static whether the activities involve motion or not,
In the duty cycles, 6s of data is needed in order to get enough respectively (Table 2); (2) Then, in the second classification, a
data so an accurate classification can be achieved. To proceed with model was built on each category so we could proceed to the
the classification we have 10s before retrieving new data. It means classification on Walking or Running on the Dynamic category,
that the data from the accelerometer does not need to be fetched all and Sitting or Standing Idle on the Static one (Table 3).
time, saving battery with less operations of the app running. To Table 2. Classifiers’ accuracy in the first level of the hierarchical approach.
sum up, we record data for 6s. Then, an instance is created with an Dynamic vs. Static Naïve Bayes Hoeffding Tree
average of the collected values. Finally, it is classified on the next Accuracy 82.11 % 99.85%
10s. This cycle is repeated along time.
Table 3. Classifiers’ accuracy in the second classification of the
hierarchical approach.
4.1 EXPERIMENTAL SETUP Naïve Bayes Hoeffding Tree
Before testing the application some decisions had to be made in Running, walking 76.25% 99.05%
order to have a controlled environment so we knew which result Sitting, standing idle 99.83% 99.93%
we were expecting for each test done.
The placement of the mobile was an important issue. Without To test the effectiveness of the classification, unlabeled data of a
having the option of placing sensors in different parts of the human person, which was not used for training the classifier, was used.
body we have chosen the trousers’ front pocket [14] to conduct all Here are the results for the walking activity – Table 4.
experiments. So there is recorded data with the mobile in a vertical
and horizontal position inside the pocket. Table 4. Accuracy for the walking activity using as test set data from a
To create the models, data from two persons was used. This data person without data on the training sets
contained the average of the values recorded by an accelerometer
for several hours doing, only, activities of walking, running, One-step Hierarchical Hierarchical
standing idle and sitting, being the waking activity the one with classification 1st classif. 2nd classif.
more recorded instances. In the total approximately 27 thousand Naïve Bayes 86,37% 90,17% 84,27%
instances were used. Hoeffding Tree 67,65% 94,04% 88,09%
The unlabeled data (files from approximately 16 thousand to 30
thousand instances) was not used to create the model. It belongs to These results only show that Hoeffding Tree is better than
the two people that contributed with data to create the model. Naïve Bayes for the walking activity on a hierarchical approach.
There is, also, data from a third person that was not used for However, Naïve Bayes gives better results on the one-step
learning the models. It was used to evaluate the semi-supervised approach (Table 4). Further tests were needed for the remaining
learning approach. activities. Additionally, a semi-supervised approach was also used,
We needed to choose between timestamp and sequence-based besides the supervised one described above, in order to evaluate the
sliding windows depending whether the window length is defined usefulness of using unlabeled data from the user that is being
according to a predefined interval or a predefined amount of data. tested.
We have chosen sequence-based sliding windows because we In order to adapt the model to the normal user of the cell phone
wanted to keep the number of instances controlled and with a time a threshold of 70% was created, as described in section 3. This
interval that is impossible because the number of data elements in
meant that data labeled with at least 70% of certainty would be
the window may vary over time.
recorded on the training file of the classifier, so a new model, more
A threshold of 70% probability is used to proceed with semi-
suitable to the user, could be generated. This approach is compared
supervised learning as explained in section 3. This allows creating
against the supervised approach (Figure 1). It is easier to check the
new models by appending to previous data the recent labeled data
better accuracy when using the semi-supervised approach.
when classified with 70% of certainty, at least.
4.2 EXPERIMENTS AND RESULTS
Previously, labeled data from three different persons was recorded.
The data contained four activities: walking, running, sitting and
standing idle. Using MOA, two different approaches were taken.
Firstly, models were induced using both Naïve Bayes and
Hoeffding Tree. The classifiers were tested on unlabeled data from
one person (Table 1).
Table 1. Classifiers’ accuracy.
Naïve Bayes Hoeffding Tree Figure 1 Accuracy of one step classification using both supervised and
Accuracy 92.00% 94.78% semi-supervised learning.
Secondly, a hierarchical approach with two levels was also After doing the hierarchical classification (Figure 2 and 3) the
carried out using the same classifiers. The hierarchical approach labeled data was checked by visual inspection and it was easy to
40
observe that Hoeffding Tree tend to label data on the first The balance characteristic of Naïve Bayes mentioned before can
classification as Dynamic (probably because the dataset is be verified in Figure 5, giving better results when used in the first
unbalanced and the Dynamic class is the majority one: there are classification. The tendency of Hoeffding Trees to classify, in the
about 15000 Dynamic instances and about 8000 Static ones). Naïve first step, the data as a Dynamic movement has influence on the
Bayes seems more balanced when labeling new data in the first second classification where Naïve Bayes has difficulties to label
classification of the hierarchical approach. data because it gets lots of Static labeled data as Dynamic data
from the first step. Overall better accuracy is achieved when using
the Naïve Bayes classifier on the first classification (Dynamic or
Static movement) and Hoeffding Tree on the second classification.
Figure 2. Classifiers' accuracy on final step of hierarchical
classification with a supervised learning approach.
Figure 5. Classifiers’ accuracy on final step of hierarchical
classification with a semi-supervised learning approach, using
different classifiers for each step.
The application had also concerns about both the battery and the
memory usage. In order to test the battery usage, a stress situation
where the app did both the hierarchical classification and the one
step classification was created. In order to do it two models were
created using the data of about 23.000 lines of labeled data, and
doing the classification of 10 unlabeled instances. This experiment
told us that the battery usage needs a maximum of 600.0mW for
the CPU and between 500mW and 600mW for the LCD, which
gives a total between 1100 and 1200mW on hierarchical
Figure 3. Classifiers’ accuracy on final step of hierarchical classification. The one step classification only creates one model.
classification with a semi-supervised learning approach. The battery usage needs a maximum of 526mW for the CPU, the
At last we tested how using the two classifiers together would LCD needs the same power as the hierarchical approach, of course.
affect the classification (Figure 4). Running the application five times, in a row, we got an energy
usage of 120.8J for the CPU in hierarchical classification. However
in one step classification we get a total of 110.3J.
Creating models and classifying about 10 instances took almost
60s which is a good time since we have only to classify 1 instance
every 16s.
In terms of memory, the prototype is about 3Mb, and the files
used for training the model having about 23000 lines are 1.466kB
each. At most we will have the existence of three files for training
(hierarchical approach). These files will grow because we defined a
limit of 30000 instances for the training set (sequence-based
window), which means that until we reach this limit none of the old
training data will be erased and new data is added. When we reach
the 30000 instances the sequence-based window will keep the size
Figure 4. Classifiers’ accuracy on final step of hierarchical of the file. Whenever new labeled data from the user arrives (using
classification with a supervised learning approach, using different the aforementioned 70% threshold) it will substitute the oldest data
classifiers for each step. in order to have a semi-supervised learning approach.
The accuracy is not the only indicator of the classifiers’
performance. Precision and recall are also important. The
technique with higher accuracy might not be the one with the best
balance between precision and recall. In our experiments we
noticed that Hoeffding Trees have a better balance between
precision and recall than Naïve Bayes.
41
5 CONCLUSIONS AND FUTURE WORK REFERENCES
The encouraging results of the experiments lead us to affirm that a [1] L. Bao & S. S. Intille (2004). “Activity Recognition from User-
step forward has been taken in the study of activity classification. Annotated Acceleration Data”, LNCS 3001, Springer, pp. 1-17.
The most difficult activities to distinguish are walking and [2] J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity
running because it is not clear where to draw the line between these Recognition using Cell Phone Accelerometers,” SIGKDD
Explorations, vol. 12, no. 2, pp. 74-82, 2010.
two activities.
[3] Tao Gu, Zhanqing Wu, Xianping Tao, Hung Keng Pung, and Jian
To achieve good results the techniques do not need to be too Lu. epSICAR: An Emerging Patterns based Approach to Sequential,
complex, like it was shown using Naïve Bayes. A fair conclusion Interleaved and Concurrent Activity Recognition. In Proc. of the 7th
after analyzing the figures is that hierarchical approach gives better Annual IEEE International Conference on Pervasive Computing and
results with Naïve Bayes doing the first classification and Communications (Percom '09), Galveston, Texas, March 9–13,
Hoeffding Tree dealing with the final one. With less complex 2009.
techniques less power of the mobile is needed, leading to a minor [4] X. Zhu, Semi-Supervised Learning Literature Survey, Tech. report,
impact on the classification performance. So, if Naïve Bayes does Computer Sciences, University of Wisconsin-Madison, USA, 2005.
not decrease the accuracy it is better to use it in order to save [5] A. C. F. Coster, “Classification of basic daily movements using a
memory and battery. triaxial accelerometer,” Medical & Biological Engineering, pp. 679-
687, 2000.
The battery usage confirms that the app can be used non-stop. It
[6] M. Schneider, M. Velten, and J. Haupert, “The ObjectRules
would be thrilling and of greater convenience to create a way that Framework - Providing Ad Hoc Context-Dependent Assistance in
could swap classification techniques when the battery was low so it Dynamic Environments,” 2010 Sixth International Conference on
could be saved and the application did not have to stop. Changing Intelligent Environments, pp. 122-127, Jul. 2010.
from hierarchical classification to one step classification would [7] N. S. Y. Wang, J. Lin, M. Annavaram, Q. A. Jacobson, J. Hong, B.
have a maximum impact of 2% on the accuracy using Hoeffding Krishnamachari, “A Framework of Energy Efficient Mobile Sensing
tree as classifier. for automatic user state recognition”, pp. 179-191, 2009.
The model used only has to be created when the application [8] B. Babcock and M. Datar, “Sampling from a moving window over
starts working. It is used for classifying until the app is shut down. streaming data,” of the thirteenth annual ACM-SIAM, 2002.
It has only to classify one instance every 16s which is enough to do [9] N. Ravi, N. Dandekar, and P. Mysore, “Activity recognition from
accelerometer data,” Proceedings of the National, pp. 1541-1546,
it, so the duty cycles work perfectly.
2005.
Regarding the memory usage a limit on the training files can be [10] G. Bieber, J. Voskamp, and B. Urban, “Activity Recognition for
created, when this limit is reached the older data can be erased and Everyday Life on Mobile Phones,” Universal Access in HCI, Part
new data added. This allows the adaptation of the application to II, HCII 2009, LNCS 5615, pp. 289-296, 2009.
new users as long as the application is being used by these new [11] P. Domingos & G. Hulten (2000). Mining high-speed data streams.
users. Proceedings of the sixth ACM SIGKDD international conference on
The application can be improved by making possible to wear Knowledge discovery and data mining - KDD’00, 71-80. New York,
the mobile on other location, testing other classifiers or changing New York, USA: ACM Press. doi:10.1145/347090.347107.
the way the data is processed. [12] T. Mitchell (1997). Machine Learning, McGraw Hill.
New tests can be made using data from people with mobility [13] Holmes, K. Richard, B. Pfahringer (2005). Tie-breaking in
Hoeffding trees. In proceedings of the Second International
constraints. Improving the app so it can adapt to this kind of people
Workshop on Knowledge Discovery from Data Streams, Porto,
can be important if an accurate prediction can be made. Studies of Portugal, 2005.
patients with diseases that tend to degrade the ability to move can [14] S. Sprager and D. Zazula, “A cumulant-based method for gait
be accomplished to prevent, for example, falls or just to study how identification using accelerometer data with principal component
the movements change. This prevention can also be applied to analysis and support vector machine,” WSEAS Transactions on
elder people. Signal Processing, vol. 5, no. 11, pp. 369–378, 2009.
With this knowledge, people who practice sport can also [15] M.M. Masud, J. Gao, L. Khan, J. Han and B. Thuraisingham,
benefit. For example, understanding how their body posture can be 2008. “A practical approach to classify evolving data streams:
corrected in order to achieve better results. Training with limited amount of labeled data”. Proceedings of the
This is just the beginning of an application that can be expanded 8th International Conference on Data Mining, December 15-19,
2008, Pisa, Italy, pp: 929-934.
in order to provide a better intimate experience between users and
[16] D. Guan , W. Yuan , Y. Lee , A. Gavrilov , S. Lee, “Activity
mobile phones. Recognition Based on Semi-supervised Learning”, Proceedings of
the 13th IEEE International Conference on Embedded and Real-
Time Computing Systems and Applications, p.469-475, August 21-
ACKNOWLEDGEMENTS 24, 2007.
[17] T. Brezmes, J. Gorricho, J. Cotrina, “ Activity Recognition from
accelerometer Data on a Mobile Phone”, Lecture Notes in Computer
This work is funded by the ERDF through the Programme
Science, 2009, Volume 5518, Distributed Computing, Artificial
COMPETE and by the Portuguese Government through FCT - Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted
Foundation for Science and Technology, project KDUS ref. Living, Pages 796-799
PTDC/EIA-EIA/098355/2008.
42