The proposed research is concerned with the problem of extending semantic models used in the IoT data-sharing contexts with new elements. These suggestions are made by the domain experts at run time and then validated by the ontology engineers. The process of terms suggestion is organized as a modeling dialogue: dynamically generated user interface forms with labels in natural language assist the domain experts in the creation of structured proposals. We hypothesize that this communication procedure will increase the e ciency and the perceived usability of performing change operations at di erent stages of the ontology evolution process. We then suggest a series of experiments to evaluate our approach.
The study by International Data Corporation \Data Age 2025" estimates a
10fold growth of annually created data in 2025; a quarter of this data is predicted
to be real-time in nature and 95% of it is expected to originate from the Internet
of Things (IoT) [
On the other hand, the data-intensive AI-based algorithms and applications
which are intended to transform the everyday lives of people and businesses,
have a critical dependency on high quality data which is di cult for them to
obtain. Sharing this IoT data can be thus bene cial for both data owners and
potential data consumers. To satisfy this increasing demand, a new type of IoT
data-sharing platform { the data marketplace, has started gaining popularity in
recent years [
Nexus, Trimble, Datum, MDM (tra c data) and Wibson (personal data).
The IoT data marketplaces trade data streams and/or aggregated sensor
data and services. These marketplaces aim at becoming a \central point of
discoverability" for IoT data. They are also required to ensure interoperability by
de ning \metaformats and abstractions" [
The creation and use of semantic models for automated IoT resource or
service discovery and subsequent data integration at run time presents two main
challenges. The rst challenge is that these models need to provide uni ed
semantics to capture not only raw sensor data but context information [
An early study of semantic interoperability in decentralized distributed
environments by Vetere and Lanzerini [
The present work takes into account the results of current European initiatives
in the eld of IoT semantic interoperability, in particular, the experience
accumulated in the BIG IoT2 project and semantic modeling solutions tested
in the IoT-Lite [
The idea of thematic templates (see below in Sec. 6) supporting the model
enrichment process extends various concepts of patterns circulating in the
Semantic Web community: \ontology design patterns" [
Another relevant research domain for the present study is ontology
evolution de ned as \timely adaptation of an ontology to the arisen changes and the
consistent propagation of these changes to dependent artefacts" [
The proposed ontology-enhanced user interface [
The research on distributed collaborative tagging systems and, in
particular, the investigation of the underlying cognitive mechanisms formalized in the
semantic imitation model of social tag choices by Fu et al. [
We understand the general approach to Semantic Web technologies-based
information sharing to be \attaching semantic meta-data to information items, and
(...) relating these metadata to each other through background knowledge in
the form of ontologies" [
The hypotheses corresponding to our research questions are the following:
H1: The organization of the semantic model extension process as a modeling dialog (coupled with data annotation process) by using an ontology-enhanced user interface where modeling primitives (basic concepts), related model fragments and similar o erings are made accessible, can simplify the process of suggesting the new concepts by domain experts.
H2: The resulting structured proposals, where concepts are linked to the corresponding super-classes and relations are established with other relevant concepts, can simplify the process of the validation and approval of new elements by ontology engineers.
H3: The proposal validation process can be, at least partially, automated by a machine learning classi er. Validation can be modeled as a single-label multiclass classi cation task where a set of categories f\Accept", \UseExisting", \ModifyAndAccept"g represent the decisions 5. The features for model training are derived from the model element characteristics (structured annotations) proposed by the domain expert, the context of a suggestion and the computed statistics of all marketplace o erings. 6
Empirical and conceptual ndings of the proposed research as well as the resulting software artifacts will be an extension of the BIG IoT project. We rst outline the relevant model design and system architecture considerations of the project and then indicate the novel components we propose.
There are three groups of the semantic modeling artefacts in BIG IoT. The Core ontology represents the concepts relevant for the marketplace operation: core:O eringQuery, core:Provider, core:Consumer, core:O ering, etc.). Two domain ontologies { Mobility and Environment { provide terms to characterize the content of the data/service o ering: mobility:ParkingSiteStatus, environment:PollutionIndicator, mobility:Tra cSpeed, etc.). Finally, the Application ontology is used for organizing navigation through the model elements in the marketplace Web portal6.
The marketplace Web portal is used to assist data providers in describing their data and data consumers in formulating queries (see Fig. 1). The user starts by selecting a category and a subcategory, and based on this initial choice, the set of available annotations for the output data is shown in dropdown lists. If users do not nd a matching semantic term, they can propose a new one by typing the concept name. 5 I owe the idea of modeling the subject expert decision as a classi cation task to Dmitry Mironov (private conversation). 6 The models follow Linked data principles and are published as schema.org custom extensions; the example terms are dereferenceable provided that the namespace pre xes can be expanded as follows: core { http://schema.big-iot.org/core/, mobility { http://schema.big-iot.org/mobility/, environment { http://schema.big-iot.org/environment/. Other pre xes in the paper are used as declared on pre x.cc
The new term is assigned a namespace pre x \proposed" and stored as a metadata annotation for the current o ering and as a potential model element. This will be later approved by an ontology engineer responsible for the marketplace models maintenance.
Addressing RQ1, we suggest the following modi cations of the new model
element inclusion process (they are summarized in the Fig. 2 above):
(1) All marketplace domain ontologies are aligned with the SSN/SOSA
ontologies [
(2) Data-driven thematic templates (we start with temporal, spatial and sensor measurement templates), are derived from a representative set of data, matching the marketplace scope, and stored as a collection of annotation graphs in a triple store (the colored layering in the sample output 1 in Fig. 2 re ects thematically di erent data).
(3) Proposing a new model element, the user rst selects a thematic template which provides a high level characterization of the proposed concept. The system, in turn, sends a request to the back-end module 6 whose main function is to retrieve a corresponding annotation graph from a triple store and to dynamically generate a series of web forms 7 based on the structure of the annotation graph (the process is exempli ed in Table 1).
(4) By answering questions displayed in the web forms, the user creates a structured semantic annotation of the new concept 8 . The annotation is persisted in the triple store as a new model element with the namespace pre x \proposed".
To test H2 corresponding to the RQ2, we will build a user interface for the ontology engineers responsible for the maintenance of the marketplace models. This interface will allow interactive exploration of 1) structured user proposals; 2) their contexts (the o ering descriptions), 3) thematic templates and existing descriptions based on them.
In our setting, we are interested in incorporating complex user-initiated changes. These changes involve the addition of new model elements, the establishing of an is-a relation with existing concepts as well as the introduction of the non-taxonomic property relations speci ed in the user annotation.
An ontology engineer may: 1) accept the proposed concept and the annotation without any modi cation, 2) decline a change request and use the existing model element instead, or 3) accept a concept but change the proposed annotation. In the cases when a change is accepted (1 or 3), it becomes a part of the model, and the e ects of this change are propagated to all dependent/related o erings and taken into account by the matching queries. When the proposed concept is declined (2), the o ering metadata is corrected accordingly.
Our preliminary results in the BIG IoT project include the alignment of the domain models with the SSN/SOSA framework, the data-driven investigation of archetypal thematic templates and the translation of their prototypical representation into basic annotation graphs. The rst version of the system architecture extension (the UI dynamic form generation and the corresponding back-end) is implemented. 7
The proposed approach and associated hypotheses will be evaluated in a series of experiments with the groups of users representing the two roles in the change management procedure: the domain experts group proposing changes and the ontology engineers group validating the proposed changes.
We have started by the construction of a gold standard dataset: 60 data samples in JSON format were annotated by two ontology engineers with a category and a data type for each key-value pair in the sample. Next, a selection of samples will be formed based on the following criteria: the data samples should be 1) typical for the intended marketplace scope; 2) balanced in terms of domains and sosa:FeaturesOfInterest ; 3) represent various thematic templates.
Then we will intentionally exclude a subset of elements representing hapax legomena (i.e. terms which occur only once in our gold standard corpus) from the existing domain models. The task formulated for a group of domain experts is to annotate given samples using the published domain models in the BIG IoT marketplace Web portal. If an annotation term is not found in the available domain models, a user is supposed to extend the model.
Experiment 1 is designed to test H1, namely, to explore a causal relationship between the ease of suggesting a new concept for data providers (dependent variable) and their exposure to the \modeling as dialogue" approach (causal variable). The control group will use the BIG IoT marketplace Web portal described in Fig. 1. The experimental group will use the implementation of the proposed approach illustrated in Fig. 2. Then the groups will be compared in terms of 1) time spent on performing an annotation task (overall and per sample) and 2) the number of times the model websites are consulted (overall and per sample). Additionally we will look at 3) the agreement with the gold standard annotation for the terms present in the model; 4) the inter-rater agreement within the groups; 5) the naming strategies used for labeling the new concepts.
We will also collect re ections of the group members on how the usability of proposing a new concept is perceived, and compare them in a structured way. For the experimental group, the average agreement with the gold standard annotation for the new terms will be quanti ed.
Experiment 2 is designed to test H2, namely, to prove a causal relationship
between the ease of validating a new concept proposal by the ontology engineer
(dependent variable) and the prior usage of the \modeling as dialogue" approach
by domain experts (causal variable). Ontology engineers will validate proposals
from the control group and structured proposals from the experimental group.
The validation process will be compared for 2 groups of proposals in terms of
1) time spent on performing a validation task (overall and per sample), 2) the
steps needed to integrate a proposed change to the corresponding domain model
(per change), 3) the steps needed for change propagation (per change); and 4)
the inter-rater agreement between ontology engineers. Questionnaires based on
the System Usability Scale [
Based on the validation results with high inter-rater agreement, a new dataset that models the decision making task will be compiled to test H3. In a machine learning experiment which aims at predicting an ontology engineer decision (dependent variable) based on the features sketched in the previous section, the standard evaluation measures of precision, recall and their harmonic mean F1 score will be used to test the model performance on a test subset. 8
The proposed research introduces the problem of run-time semantic model extension by a domain expert mediated by the ontology-enhanced user interface and further validated by an ontology engineer. Its main contribution is intended to be the experimental and theoretical basis for extending semantic models. These models will serve both as descriptive metadata for the resources exposed on an IoT data marketplace and also as a lingua franca (shared model) in the infrastructure where data sharing takes place.
Through the exploration of various scenarios of on-the- y model extension performed by data providers, we will develop methods and tools to optimize the new model element inclusion process, paying special attention to the aspects which can contribute to the partially automated work ow. Although our research is shaped by and tailored to the IoT data marketplaces context, we believe that the approach described above is generic and can be applied in many scenarios that require model usage and extension by non-experts.
Acknowledgements. This work has been developed in the project BIG IoT funded by the European Commission's Horizon 2020 program under the grant agreement No 688038. The author would like to thank Prof. Dr. York Sure-Vetter (the thesis advisor), Prof. Dr. Andreas Harth, Dr. Stefan Schmid and three anonymous reviewers for their critical reading of the paper, their constructive comments and suggestions.