One of the most critical challenges during critical incidents in big public events are the absence of real time information as well as the lack of ltering this information to meaningful knowledge in order to provide crisis decision support. Towards addressing those challenges, semantic technologies can provide a semantically data representation of the critical events, along with superior capabilities in reasoning to support the decision making. In this paper we describe the semantic reasoning framework of the DESMOS project, that could constitute the backbone of the decision support systems in public events.
The era of the Internet of Things (IoT) is upon us, with a huge number of IoT devices already in everybody lives. A large number of applications in Smart Cities, E-health, Security and other domains are exploiting the IoT technologies, like sensors, smartphones and actuators. One of the most signi cant domains of IoT is the safety and security, which IoT is used to save lives and mitigate dangerous incidents.
However, several challenges arose with the existing IoT technologies regarding the interoperability of those di erent IoT technologies since their data is based on proprietary formats and they do not use common formats or a vocabulary to describe the interoperable data. The basic structure of the IoT is the Machine-toMachine (M2M) communication. For example, the measurements of sensors are required to be distributed and analyzed by other devices or sensors and not being human readable without any kind of processing. Therefore, those measurements should be understandable from one machine to another.
In this paper, we describe the methodology and procedure that underpin the development of reasoning rules in the DESMOS ontology. DESMOS is a novel framework for the intelligent interconnection of smart infrastructures, mobile and wearable devices and apps for the provision of a secure environment for citizens, especially for visitors and tourists. The platform aims to promote the collaboration between people and devices for protecting tourists, supporting timely Copyright c 2020 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0). reporting of incidents, adaptation of the interconnected environments in case of emergency, and the provision of assistance by empowering local authorities and volunteers. The main research contribution of the paper is the combination of the data processing modules (localization) with the semantic knowledge under a common reasoning framework.
The rest of the paper is organized as follows. In Section 2, we describe related work. In Section 3 the Desmos framework and the main technical components. Section 4 describes the Data Integration. rule-based semantic reasoning. In Section 5 the Rule-based Reasoning is presented. Finally, Section 6 concludes the paper. 2
The Internet of Things (IoT) is becoming more and more popular and its applications are facing an enormous proliferation resulting in a new digital ecosystem. IoT is based on a wide range of di erent heterogeneous technologies and devices, there is not a uniform vocabulary for representation and processing of data. This has led to a large number of incompatible IoT platforms. Through this way, it is very di cult for data scientists to extract knowledge from the enormous number of data producing every second through the IoT applications.
Semantic web technologies tries to overcome such challenges. Semantic web leverage web standards and semantic technologies to interconnect all types of devices by transforming low-level sensor data into high-level knowledge that is comprehensible to humans and machines. Semantic modelling produces a de nite description of the data meaning in a structured way by combining application knowledge and context-relevant information with sensor data. The ontology based development, which is a domain of semantic modelling, of IoT frameworks can lead to universal IoT solutions multiplying the bene ts of IoT.
W3C (World Wide Web Consortium) suggested an ontology, the
Semantic Sensor Network (SSN), as a human and machine-readable speci cation that
covers networks of sensors and their deployment on top of sensors and
observations [
One of the most important ontologies for event annotation is Event [
There are many research studies for semantic reasoning on critical event
response and management. In [
In this section we are presenting the framework for encoding, aggregating and semantically analysing information relevant to the DESMOS application domain. The framework is presented in Figure 1. It consists of a) a Communication Layer that allows the interactions between the system components, b) an Ontology, that is used for semantically representing notions pertinent to the project, c) a Knowledge Base Repository (KBR) that hosts the domain ontology, d) a Knowledge Base Service that acts as the main interface to the ontology, responsible for populating the ontology with incoming data as well as performing rule-based reasoning over the knowledge base, e) Mobile Apps, Wearable Devices and Sensors that feed the system with real-time data and nally f) a Localization Module that retrieves the received signal strength indicator (RSSI) values of the wearable devices from the KBR in order to calculate the distance based on data analysis algorithms. 3.1
The basis for the development of an intelligent interconnected framework for public security and protection are the needs and requirements of the domain experts. The purpose of the requirements collection is to understand the needs of the users and the problems they try to solve. A common methodology for de ning the use cases and requirements has been used, starting from the identi cation of the most common security and protection incidents in the pilot location, the ways of dealing with them up to now, and the di culties encountered in managing them.
KNOWLEDGE BASE REPOSITORY
Mobile Apps
Wearables
Locati on
The use cases show in Table 1 were identi ed and studied by running several focus group sessions between stakeholders concerned with integrated risk management (municipal authorities, volunteers etc.) focusing on their needs. In addition, requirements were identi ed and prioritized through interview questions and focus group sessions between domain experts, end-users, lawyers and software engineers. Table 2 contains an indicative subset of the user requirements. These user requirements are catalogued as [UR xzz ], where x is the identi er of the use case scenario in which the requirement originated (see Table 1), and zz is the serial number of the requirement. 4
The Knowledge Base Service (KBS) in Figure 1 is a central component of the system architecture since it is the main interface to the DESMOS ontology also referred to as the Knowledge Base (KB). The ontology is the knowledge representation model for semantically representing notions pertinent to the project: (a) location of the personnel, (b) location of automated external de brillators (AED) (c) the RSSI observations made by mobile devices or xed nodes (i.e. hubs) both listening for broadcasted addresses from Bluetooth Low Energy (BLE) devices d) the results of the RSSI analysis service (i.e. Localization module), e) visitor alert requests and f) personnel assignments to critical incidents.
Hub
subclassOf Coordinates coordinates Location Sensor contains subclassOf Sensor BLE rssi RSSI mac
Agent owns Device subclassOf Transmitt
er MAC subclassOf
Human hasRole
Point Irsse u lft o
Role Visitor subclassOfsubclassOf
Volunteer Notification subclassOf Action requires
Event Trigger
Beacon takesAction location Volunteer Observation
Visitor Notification category subclassOf Health
Issue triggers contains
Category subclassOf
Loss subclassOf Missing Person
GEO
FOAF
Data DESMOS SSN/SOSA have been reused. In addition, KBS features subscribe/publish capabilities to a dedicated communication bus (MSB) allowing interactions of the KB with other system components. In a sense, KBS accepts input from other components and semantically integrates it into the ontology. It also handles the output of the semantic reasoning process running on top of the KB and forwards the inferred, high-level knowledge back to other interested system components.
As shown in Figure 1, KBS consists of two sub-components, namely the Knowledge Base Population (KBP) component, responsible for integrating data into the KB, and the Events Crawler that is activated as soon as an event is triggered. The interactions of the KBS with the KBR is achieved through SPARQL queries or by using the RDF4J2 library tools.
More speci cally, the communication layer allows the exchange of messages in JSON format. These messages are analyzed by the KBS and are semantically integrated into the appropriate classes considering the domain ontology. The ontology itself as well as the data collected from the various system components (e.g. sensors, mobile apps etc.) will be stored in the KB and the KBP component is responsible for the data manipulation. Some indicative examples of representing notions of the domain are given below (i.e. in RDF Triples). The rst example depicts the representation of a Volunteer along with his location consisting of the Volunteer's latest coordinates (i.e. latitude and longitude).
Listing 1.1. RDF representation of a Volunteer in the KB. <volunteer/867325031117405/> rdf:type desmos:Human ; desmos:hasRole desmos:Volunteer ; desmos:id "867325031117405"^^xsd:string ; desmos:name "Volunteer_1."@en ; sosa:hosts <device/867325031117405> ; rdfs:label "Volunteer 1"@en . geo:hasGeometry [geo:asWKT POINT(22.9954257 40.5654389)]
Moreover, mobile devices belonging to Volunteers send regular messages about registered wearable devices that are within their range. These messages are also characterized by a unique identi er of the sending device that will allow the information on the wearable device to be matched with the mobile device that sent the message. Below is the representation of an rssi observation in the ontology. Note how the last property in the representation, isObservedBy, matches the corresponding observation with the mobile device.
Listing 1.2. RDF representation of an RSSI Observation. <observation/E7:4E:C3:10:B4:D0/Obs_1> rdf:type sosa:Observation; sosa:observedProperty
<sensor/E7:4E:C3:10:B4:D0/powerSignalSensor/RSSI>; sosa:hasFeatureOfInterest <powerSignal/E7:4E:C3:10:B4:D0>; sosa:madeBySensor <sensor/E7:4E:C3:10:B4:D0/powerSignalSensor>; 2 https://rdf4j.org/ sosa:resultTime "2019-05-29T12:52:01Z "^^xsd:dateTime; sosa:usedProcedure <MeasuringPowerSignal>; desmos:signal_strength "-51"^^xsd:double; sosa:isObservedBy <device/867325031117405>.
In addition, Visitors of the park may use the application that they have previously installed in their mobile devices and send a request that will trigger an Event. This request may refer either to a Health Issue or a Missing Person situation. The following example depicts the representation of a Health Issue event in the ontology.
Listing 1.3. RDF representation of a Health Issue Event. <event/0a990cf5-8c3f-4484> rdf:type desmos:HealthIssue; desmos:id "0a990cf5-8c3f-4484"^^xsd:string ; geo:hasGeometry [geo:asWKT POINT(22.9879934 40.57327)]
It is worth mentioning that there is an initialization phase where the ontology is populated with the volunteer pro les together with the necessary information about their mobile devices, the exact location of the xed nodes that listen for broadcasted addresses from BLE devices as well as the location of the AED systems that are available in the area. Once this data are populated in the ontology, the KBS can perform reasoning to infer new knowledge, and then update the interested components. 5
In addition, in order to search for incoming messages and handling their semantic representation and integration into the KB, KBS also incorporates a reasoning mechanism to infer underlying knowledge and discover connections between ontology entities during various events occurring in space, such as the disappearance of a person, or various medical incidents. This mechanism is rulebased and consists of a set of SPARQL queries. These queries were constructed by following the approach that is described in the following subsection. 5.1
The user requirements in Table 2 were rst mapped to ontology's Competency
Questions (CQs). A competency question is a natural language sentence that
expresses a pattern for a type of question people expect an ontology to answer
[
Given a Visitor's request request that is related to UC 1 (i.e. request for medical help) and consists of a request id and the coordinates of the user requesting for help, the following queries (Queries 1.4, 1.5) calculate the nearest Volunteer and the nearest AED respectively.
Listing 1.4. (UC 1): Who is the nearest Volunteer? SELECT ?imei WHERE { ?not rdf:type desmos:VisitorNotification ; geo:hasGeometry [ geo:asWKT ?point1 ] ; desmos:id ?not_id. ?volunteer desmos:hasRole desmos:Volunteer ; geo:hasGeometry [ geo:asWKT ?point2 ] ; desmos:imei ?imei.
BIND((geof:distance(?point1, ?point2, uom:metre)) as ?dist) . FILTER(?not_id = STR(not_id)) . } ORDER BY ASC(?dist) LIMIT 1
Listing 1.5. (UC 1): What is the nearest AED? SELECT ?aed_id WHERE { ?not rdf:type desmos:VisitorNotification ; geo:hasGeometry [ geo:asWKT ?point1 ] ; desmos:id ?not_id. ?aed rdf:type desmos:AED ; geo:hasGeometry [ geo:asWKT ?point2 ] ; desmos:id ?aed_id.
BIND((geof:distance(?point1, ?point2, uom:metre)) as ?dist) . FILTER(?not_id = STR(not_id)) . } ORDER BY ASC(?dist) LIMIT 1
Similarly, given a Visitor's request that is related to UC 2 (i.e. request help for lost child) and consists of the request id and the mac address of the child's wearable device, the following queries (Queries 1.6, 1.7) are calculating the latest observers of the missing child (i.e. Volunteers) and their distance from the lost child respectively. More speci cally, query 1.6 nds the 3 nearest observers while query 1.7 uses the latter information as input (i.e. through FILTER) and calculates the respective distance of each observer.
Note that the distance between the Volunteers and the lost child is calculated based on RSSI-based localization techniques that are performed by the Localization Module. More speci cally in UC 2 children are wearing wearable equipment emitting a Radio Frequency (RF) signal with an RSSI that is received from the the volunteers' devices as well as from several xed points (i.e. hubs) scattered around the pilot area. The RSSI values are then used to calculate the respective distance in meters (in the Localization module) and this information is integrated into the KB and enriches the existing knowledge.
The localization module analyzes the RSSI values based on a kalman ltering and a regression method that transforms the RSSI values to distances. After that, given the coordinates of the closest Volunteers and Hubs from the KB, namely A; B; C with coordinates (x1; y1); (x2; y2); (x3; y3) respectively, whose distance from the point of interest (i.e. lost child) is d1; d2; d3 respectively which was estimated before, we can then calculate the coordinates of the point of interest (x; y) using the following equations:
Listing 1.6. (UC 2): Who are the child's latest observers? SELECT ?imei (MAX(?time) as ?instanceTime) { ?s rdf:type sosa:Observation ; sosa:isObservedBy ?o ; desmos:isDataOf ?p ; sosa:resultTime ?time . ?vol sosa:hosts ?o ;
desmos:imei ?imei . ?p desmos:mac ?mac .
FILTER(?mac = STR(mac)) } GROUP BY ?imei ORDER BY DESC(?instanceTime) LIMIT 3 Listing 1.7. (UC 2): What is the location of the lost child's observers and their respective distance from the lost child? SELECT ?imei ?lat ?long ?distance { ?s rdf:type sosa:Observation ; sosa:isObservedBy ?o ; desmos:isDataOf ?p ; sosa:resultTime ?time ; desmos:distance ?distance . ?vol sosa:hosts ?o ; desmos:imei ?imei ; desmos:lat ?lat ; desmos:long ?long . ?p desmos:mac ?mac .
FILTER(?imei = STR(imei) && ?time = time) } 6
In this paper, we have discussed and demonstrated an ontology-based reasoning framework for the management of critical incidents in public events. We have presented the main components of the DESMOS semantic reasoning framework and how this has been incorporated in the DESMOS main framework. For future work, the framework will be tested and validated in two pilots in two di erent public places which protection and security challenges are higher. 7
This research has been co- nanced by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH-CREATE-INNOVATE (project code: T1EDK-03487).