IoT devices in Smart homes generate large amounts of data by sending messages to share measurements at frequent intervals. To make this IoT measurement data interoperable, ontologies such as SAREF are used to represent this data as knowledge graphs (KGs). These resulting KGs contain measurements from IoT devices together with the context about those devices, such as the type of measurements, or in which room it is located. Through the use of embedding methods, we learn embedding representations for entities in KGs that can be used to generate prediction models. In this study, we investigate how the structure of the IoT knowledge graph afects the efectiveness of embedding methods for such tasks. Specifically, we look at the efect on embedding quality of enriching IoT KGs by making implicit information such as temporal sequences and measurement value similarity explicit. We perform experiments on two IoT KGs represented using the SAREF ontology. One is in a basic setting, and the other has this information explicitly added. We use RDF2vec to create embeddings, train a classifier, and compare classifier accuracy.
With IoT devices becoming more prevalent in everyday homes, common frameworks in which
IoT devices can interact become more important. [
All this information combined in one KG provides opportunities to learn over the shared knowledge by using embedding methods that create vector representations of the entities in a KG, which can be used to train ML models used as forecasters or classifiers. A classifier could, for example, be used to predict whether the outside temperature is expected to be warm or cold, which in turn can be used to turn a heater on or of to save energy.
However, the ontologies that are used in these frameworks were instead developed to optimize
embedding methods to learn from them, they were developed in a manner that: “facilitates the
matching of existing assets in the smart applications domain”. [
Before the experiment is described in more detail, the next section will give an overview of relevant research and use it to define some concepts we used in this work.
In this section, we will give a short overview of existing work that is relevant to our research. The first part explains what we define as IoT KGs, while the second part provides background to embedding methods and specific methods that will be used in our experiments.
We define IoT KGs as KGs specifically created to represent measurement data from IoT devices.
SAREF [
As seen in Figure 1, there are a few entities, such as s a r e f : D e v i c e or s a r e f : P r o p e r t y , that are
connected to the much larger amount of measurement entities. This disproportionate imbalance
between many measurement entities, and a few other entities is what we consider to define
a IoT KG. As stated in [
When examining the Web of Things ontology a similar structure can be observed, with the
w o t : p r o p e r t y entity acting as the s a r e f : M e a s u r e m e n t entity. [
SAREF:Task SAREF:FeatureOfInterest
SAREF:Property
SAREF:UnitOfMeasurement SAREF:Measurement1
SAREF:Measurement2
SAREF:Measurement3
Network ontology uses s s n : o b s e r v a t i o n . [
The purpose of embedding models is to create a numerical representation for specific entities.
Word2vec learns embeddings for words based on with which other words they co-occur in
sentences. [
In [
Adding new information to a graph based on implicit information can have an adverse efect,
as shown by [
Step I
Embedding Model Step II Step II
Classification Task Step III Step III
compare
implicit information, but the quality of the embeddings actually declines. They hypothesize that the initial absence of the implicit information was in itself a signal. Our semantic enrichment approach similarly utilizes implicit knowledge, but in the case of IoT KGs, the implicit information is used to create new relations because the current relations are insuficiently connecting relevant entities.
In order to expose the efect that semantically enriching IoT KGs has on entity embeddings, the following experiments were created. The experimental pipeline is depicted in Figure 2. Each experiment consists of two graphs: an IoT graph (basic graph), and a copy of that first IoT graph that has been semantically enriched (enriched graph). For the second step an embedding method is used to create embeddings for specific entities in each graph, and step three uses these entity embeddings to train a classifier. Comparing the accuracy of the classifier trained with embeddings from the basic graph, with the accuracy of the classifier trained with embeddings from the enriched graph will show the efect of the semantic enrichment.
Each step is described in more detail in the following subsections. All the code used in these experiments is available in our GitHub repository.2
To investigate whether the amount of devices afects the embeddings, the experiments were performed with three diferent quantities of devices. We used either all the measurements of: one device, all the devices from one home, or all the devices from all the homes in the dataset.
In order to perform the semantic enrichment of the graph we created three new entities based on implicit knowledge in the graph.
Rounded value (red line in Figure 3) Each measurement entity has a measurement value, which is used to create a new entity based on the rounded value of the measurement. In this 2https://github.com/RoderickvanderWeerdt/semantic-enrichment-of-IoT-graphs SAREF:Device
SAREF:Task SAREF:FeatureOfInterest
SAREF:Property
SAREF:UnitOfMeasurement SAREF:Measurement1
For the embedding step of our experiment, we use the pyRDF2vec implementation of RDF2vec.
[
An initial experiment compares diferent settings for the following hyperparameters: number of walks per entity, length of the walks, and number of epochs the embeddings are trained. A subset of the dataset described in Section 3.4 was used that only included 2000 time points, to decrease the time it takes to train the embedding model. Based on the results of this first experiment settings were selected to run the remaining experiments.
The evaluation of the embeddings quality is a classification task, classifying each timestamp as either hot or cold. The timestamps were labeled by dividing the dataset in two by sorting the timestamps from hot to cold and labeling the first half as warm, and the second half as cold.
The classification was performed with a Multilayer perceptron (MLP) implementation written using PyTorch [11], consisting of two hidden layers with 512 ReLU activation nodes. These were used as the default hyperparameters. The input to the MLP was the embeddings of the timestamp, and the output was the hot or cold label.
At the moment of writing, we were not able to find open IoT KGs, which is likely due to the privacy concerns of making people’s IoT device data publicly, therefore we created these ourselves, using an energy consumption dataset on device level containing real device measurement data. We created the IoT KGs following the mapping procedure detailed in [12]. Templates were created for each device in the dataset, mapping each measurement to the graph together with related properties.
In order to create the entity files for the embedding step the timestamps of a dataset were collected and used to connect with the outside temperature at that location at the specific time. From those temperatures, the hot and cold labels were created. The outside temperature values were retrieved through worldweatheronline.com.
OPSD Household data The OPSD Household 1hour dataset [13] consists of energy consumption measurements from devices in diferent types of buildings, of which we use only the six residential homes since this is closest to the type of measurements we want to investigate. Table 2 shows the distributions of the diferent device types over each separate residence. 3
The dataset contains measurements taken over a five years duration, but not every device recorded measurements for the entire period. In order to have a complete dataset we chose to extract a subset of ten months where all devices had recorded measurements, by removing only two devices (the freezer from residence 2, and the grid export from residence 6), if we would have included these the measurements would only be available for two months.
The final manipulation of the data was transforming the energy consumption measurement from its original value of accumulated consumption from the startpoint, to accumulated consumption over the last hour. This manipulation was performed to ensure that the measurement values in the graph would be recurring, which would not be the case for accumulated measurement values because those would only increase.
The final graph represents 8133 timestamp entities linking to measurements from 37 devices from ten device types, spread out over six residences. Three diferent versions of this graph were created in order to be able to distinguish between the efects of adding more devices from within the same home, and adding devices from other homes. The following shorthand is used to refer to diferent compositions of the graph:
res1dev1: this graph uses only measurements of one device. this is the heatpump from residence 4. This graph contains 89477 triples.
res1devA: this graph uses all measurements from all devices in one home, in this case, all devices from residence 4. This graph contains 715,765 triples.
resAdevA: this graph uses all measurements from all devices in all available homes that are available. It contains 3,220,912 triples. 3The IoT KGs can be found at: https://github.com/RoderickvanderWeerdt/SAREFized-OPSD-household-graph Semantically Enriched data After the semantic enrichment described in Section 3.1 was performed on the OPSD dataset the KGs grew: res1dev1: This graph contains 122,007 triples. res1devA: This graph contains 976,005 triples. resAdevA: This graph contains 4,391,992 triples.
In this section, the results of the experiments are reported. The hyperparameter selection uses the subset with the 2000 hottest and coldest time points of the OPSD dataset, while the full OPSD dataset is used in Section 4.2.
The intention of the hyperparameter selection experiment was to find a combination of short training time and ideally accuracies that were representable for the quality of the embeddings learned from each graph, not to optimize for the highest accuracy possible.
Table 3 displays accuracies and training times achieved with the resAdevA-2000 KGs, for multiple hyperparameter settings. Each combination was trained for 40 epochs.
After removing the options that were ended prematurely due to an expected training time of at least 50 hours, marked here with a ‘x’, the grid was reduced into two directions to explore, increasing either the number of walks or the walk length. By increasing the walk length the diference in accuracy between using the basic or enriched graph grows, but the training time also grows extensively. Alternatively increasing the number of walks had a similar efect on the accuracy, but the time to train only grew slightly between diferent settings.
We attribute this diference in training time to the increase of entities that were needed to be retrieved during training of the models with an increased walk length. As discussed earlier in Section 2.1, IoT KGs require only a few steps through the graph to reach all entities in the graph. By increasing the walk length more entities are included that are connected to many entities, increasing the possible unique walks greatly. To handle this more compute time is required.
Based on our requirements of short training time and distinct accuracies the best option is a walklength of 2 and 25 walks per entity, since the time increase is limited, and the accuracy increase allows for more range between the embedding quality of basic and enriched IoT KGs.
In order to determine the efects of the number of epochs the embeddings model is trained, we trained six embedding models, one for each version of the basic and the enriched IoT KG. The training and test accuracy were calculated after each epoch, for a total of 40 epochs. The results are visualized in Figure 4. Training accuracies are portrayed with dashed lines, test accuracies with solid lines.
All training accuracies reach 100% after 10 epochs, except res1dev1-basic, which takes 20 epochs. Two out of three test accuracies for basic IoT KGs never increase above ±50% and are therefore not taken into account. The enriched IoT KGs test accuracies remain consistent after 20 epochs.
Based on these findings we trained the model for the next experiment for 20 epochs, this is enough to reach the plateau of resAdevA enriched and res1devA enriched, but will also avoid overfitting by not training too long.
The results of the experiment have been visualized in Figure 5. For both datasets and for any amount of devices being used, the classifier trained on the enriched graph always outperforms the classifier trained on the basic graph. The smallest diference occurred with res1dev1 of the OPSD data, and even then the accuracy was still significantly diferent ( p=0.00766).
Accuracies of classifiers trained on basic IoT KGs were, with the exception of res1dev1, always close to 50%. This indicates that these embeddings did not contain any information that the classifier was able to learn from since only two possible classes exist and therefore always picking the same class yields an accuracy of 50%. The exception being the classifier trained on the res1dev1 graph. A possible explanation is that because this is a singular device, all measurements values that were available as literals were enough for the embedding model to learn a signal because that one device by itself happens to be a very good indicator for what was being classified, the energy consumption of a heatpump to predict the outside temperature.
Accuracies of classifiers trained on enriched IoT KGs were highest when learning from res1dev1. As with the classifiers trained on basic IoT KGs, this could be explained by the fact that this device is a good indicator for the classes. When more devices from the same residence were added (res1devA) the accuracy of classifiers reduced, which might be explained by all the additional devices, and therefore measurements, that were added, are not providing more, or the right, information for the embeddings, causing the classifiers accuracy to decrease. For example, a dishwasher would not be a good predictor of the outside temperature.
Classifiers trained with embeddings from resAdevA had a significantly higher accuracy (p=0.00099) than classifiers trained with embeddings from res1devA. The diference in relation to res1devA and res1dev1 being that not only other devices were added, but other devices from diferent homes were added, which include new devices that are good indicators for the classification labels.
During this research, some unexpected observations were made, which are presented in this section. They serve as a way to explain our process, but also to suggest some possibilities for future research.
For now, only energy consumption measurements were used, primarily because this was the only public data we could acquire at the time. More heterogeneous measurements could have a diferent efect of the embedding methods. Additional measurements could for example include CO2 levels or room occupancy.
In Section 2.1 we explained why we expect our results to generalize to IoT KGs created with diferent ontologies, based on the similarities in their design. Additional research could explore the diferences, to determine if semantic enrichment has a bigger or smaller efect.
The rounded values were now always rounded to one decimal, using more decimals would result in more rounded value entities, each having fewer connections with measurements, since fewer measurement values would be rounded to the same rounded value. But rounding to too many decimals runs the risk of becoming too specific, with a “rounded” value for every measurement value. How much rounding is applied could be an interesting parameter for future research.
Even though all the measurements were about energy consumption, they were made by many diferent devices. But the rounded values (and literal values) made no distinction between, for example, a measurement of 0.6 of a freezer, or a measurement of 0.6 of a dryer. Making separate entities out of these, such as 0 . 6 _ f r e e z e r and 0 . 6 _ d r y e r would allow the embedding method to learn the diference.
For now, we used one classifier without any optimization. For the purposes of this experiment that suficed to compare the accuracies of the classifiers trained on diferent IoT KGs, but other classifiers could be used to examine if they are diferently afected with diferent IoT KGs.
The classifiers are overfitting on the training set, as can be observed in Figure 4. This could be an indication that we need more data, or that the embeddings are creating too specific representations for the entities. RDF2vec not only embeds similar entities close together, but it also tries to embed dissimilar entities further away, which could cause all the entities to be spread out evenly, leaving no generalization to learn for the classification model. Future research could reimplement this experiment with diferent embedding methods, such as TransE [14] or GCN [15], which could make the diference in accuracies achieved in the experiment more clearly distinct.
In our experiments, we predicted the outside temperature based on a house, but this outside temperature is the same for all the houses in each dataset because we only have one outside temperature on city level. Therefore the result that including more houses leads to a higher accuracy can also mean a more representative house was added, that better represents what is being classified. Future research should explore the benefits of sharing information between diferent homes to improve embeddings, but also explore how this afects privacy concerns.
In this research, we set out to answer the question: What is the efect of semantically enriching a KG on the quality of entity embeddings learned from it. When we consider the accuracy of the classifier as an indicator of the quality of the entity embeddings, the results of our experiment make it clear that semantic enrichment had a positive efect on the entity embedding quality. The semantically enriched IoT KGs outperformed their corresponding basic IoT KGs in every instance, regardless of graph size.
Because the information added by the semantic enrichment is implicitly already available in the KG, changing the ontology design to add the enrichments would not provide any new information to the ontology. Instead, we see the semantic enrichment step as an additional part of preprocessing the graph, to enhance the quality of the embeddings.
Acknowledgements. This work is part of the InterConnect project (interconnectproject.eu/) which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 857237. [11] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, Pytorch: An imperative style, high-performance deep learning library, in: Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019, pp. 8024–8035. URL: http://papers.neurips.cc/ paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf. [12] R. van der Weerdt, V. de Boer, L. Daniele, B. Nouwt, R. Siebes, Making heterogeneous smart home data interoperable with the SAREF ontology, International Journal of Metadata, Semantics and Ontologies 15 (2021) 280–293. [13] Open Power System Data, Data Package Household Data, Version 2020-04-15 https://data. open-power-system-data.org/household_data/2020-04-15/. (Primary data from various sources, for a complete list see URL)., 2020. [14] A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational data, Advances in neural information processing systems 26 (2013). [15] T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv:1609.02907 (2016).