=Paper= {{Paper |id=Vol-2739/paper_2 |storemode=property |title=Integrated Solution for Industrial IoT Data Security - the CHARIOT Solution |pdfUrl=https://ceur-ws.org/Vol-2739/paper_2.pdf |volume=Vol-2739 |authors=Konstantinos Loupos,Alexandros Papageorgiou,Thomas Krousarlis,Antonis Mygiakis,Konstantinos Zavitsas,Christos Skoufis,Stelios Christofi,Vasos Hadjioannou,Sofiane Zemouri,Magdalena Kacmajor,Andrea Battaglia,Andrea Chiappetta,Jacopo Cavallo,George Theofilis,Harris Avgoustidis,Vassilis Kalompatsos,Basile Starynkevitch,Franck Vedrine |dblpUrl=https://dblp.org/rec/conf/sam-iot/LouposPKMZSCHZK20 }} ==Integrated Solution for Industrial IoT Data Security - the CHARIOT Solution== https://ceur-ws.org/Vol-2739/paper_2.pdf
 Integrated Solution for Industrial IoT Data Security –
               The CHARIOT Solution
     Konstantinos Loupos, Alexandros                     Sofiane Zemouri, Magdalena Kacmajor                Harris Avgoustidis, Vassileios Kalompatsos
 Papageorgiou, Thomas Krousarlis, Antonis                           IBM Ireland Ltd,                                      TELCOSERV,
                Mygiakis                                           Ballsbridge, Ireland                               Agios Stefanos, Greece
           Inlecom Innovation,                                sofiane.zemouri1@ibm.com,                             {h.avg, vkal}@telcoserv.gr
             Athens, Greece                                magdalena.kacmajor@ie.ibm.com
  {name.surname}@inlecomsystems.com
                                                      Andrea Battaglia, Andrea Chiappetta, Jacopo               Basile Starynkevitch, Franck Vedrine
  Christos Skoufis, Stelios Christofi, Vasos                            Cavallo                                              CEA, LIST,
                  Hadjioannou                                        ASPISEC Srl,                                      Gif-sur-Yvette, France
          EBOS Technologies Ltd,                                      Rome, Italy                                     {name.surname}@cea.fr
                Nicosia, Cyprus                               {a.battaglia, a.chiappetta,
  {christoss, stelios, vasosh}@ebos.com.cy                     j.cavallo}@aspisec.com

             Konstantinos Zavitsas                                   George Theofilis
                 VLTN GCV,                                            CLMS Hellas,
             Antwerpen, Belgium                                       Athens, Greece
             kzavitsas@gmail.com                                 g.theofilis@clmsuk.com



    Abstract— The CHARIOT H2020 (IoT) project (Cognitive                                    II.    INDUSTRIAL IOT SECURITY ORIENTATION
Heterogeneous Architecture for Industrial IoT), integrates a state-
of-the-art inclusive solution for the security, safety and privacy                A. Industrial Requirements Overview
assurance of data in industrial networks. The solution is based on                    The requirements related to the CHARIOT project offerings
an integrated approach for IoT devices lifecycle management                       are strongly related to recent challenges in modern IoT networks
(based on blockchain and public key infrastructure technologies),
                                                                                  and mostly target sensing and monitoring systems in various
IoT firmware development and deployment (source and binary
                                                                                  industrial themes including smart buildings, airports and trains.
level vulnerability analyses), data analytics (privacy by design,
sensitive data detection, dynamic network configurations etc.) and
                                                                                  All investigated scenarios require data exchanges in a safe,
a set of user interfaces for management and control of the                        secure and private approach resulting into overall needs of
network, devices and the CHARIOT platform. CHARIOT is                             trusting the actual sensors and information they convey in a
funded by the H2020 programme under the IoT topic, has a 3-year                   complex network, guaranteeing thus the network devices
duration and concludes its activities by the end of 2020.                         accuracy and non-intrusion. These challenges have driven the
                                                                                  CHARIOT solutions in placing the actual network devices as the
    Keywords— IoT, industrial data, security, privacy, safety                     ‘root of trust’ in these IoT networks [1] [2] [3].

                           I.    INTRODUCTION                                         CHARIOT central revolution and innovation over the
                                                                                  current state of the art is oriented in placing the actual devices of
    The CHARIOT project is focusing its activities on an                          an IoT network as the root of trust through its cohesive approach
integrated solution towards recent risks and challenges of the                    towards Privacy, Security and Safety (PSS) of industrial IoT
industrial IoT domain. These include a wide span of cyber                         Systems. This is achieved through a combination of Public Key
technological concerns and attacks that include: i)                               Infrastructure (PKI) technologies coupled with pre-programmed
eavesdropping, interception and hijacking (man in the middle,                     private keys deployed to IoT devices with corresponding private
protocol hijacking, network reconnaissance etc.), ii) Nefarious                   keys in Blockchain for affirming/approving valid transactions, a
activities, abuse (malware, denial of service, software                           blockchain       ledger      affirming      various     levels     of
manipulation, targeted attacks, personal data abuse and brute                     operational/functional changes in the network (devices
force attacks), iii) unintentional damages (configuration                         authorization, provisioning, status changes etc. as an audit log),
changes, third party damages, erroneous usage etc.), iv) network                  a supervision engine combining supervision, analytics and
failures and malfunctions (failure of sensor/device, software                     predictive modelling over IoT data and a firmware development,
vulnerabilities, failure/malfunction of control systems) and v)                   validation and update approach (based on online and offline
legal (contractual requirements, violation of rules). The paper                   code/binary analyses) securing end-to-end code development
contribution is summarized to IoT Devices’ Lifecycle                              and execution on the devices.
management, IoT Firmware Development and Deployment,
Intelligent IoT Data Analytics and IPSE and Platform and User                       CHARIOT provides a series of unique and innovative
Interface as components of the CHARIOT solution.                                  management features for Industrial IoT and connected devices



Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

                                                                             11
including providing devices’ software and firmware level                     where collected data are analysed in modern systems to perform
security and sensor visibility through a dashboard for,                      continuous monitoring of traffic flows, prevention, early
configuration, software updates management etc. By automating                detection, diagnosis and mitigation of the data breaching effect
key sensor management functions using blockchain, PKI and                    controlling the IoT sensors data package that are delivered to
automated workflows, CHARIOT provides a solution to coping                   Dynamic Maintenance Management Systems. In this case, train
with the fast pace growth of emerging IoT technologies whose                 operators need a system that checks the IoT communications and
pace of evolution is faster pace than skilled staffing and                   collects status reports informing the operator of potential
available resources while at the same time places the IoT devices            security violations detected.
as the root of trust (central innovation point in CHARIOT). In
other words, CHARIOT automates key sensor management                              III.   OVERALL CHARIOT TECHNICAL ORIENTATION
functions to improve their cost effectiveness. In this direction,               In view of detailed analyses of the above requirements,
CHARIOT, addresses the whole lifecycle of IoT devices and                    CHARIOT is developing an innovative Privacy, Security and
networks supporting various verticals.                                       Safety (PSS) platform for IoT Systems, that places devices and
B. Building Management Requirements and Challenges                           hardware at the root of trust, in turn contributing to high security
                                                                             and integrity of industrial IoT.
    In building management view, CHARIOT has investigated
the IBM Technology campus (partner in CHARIOT) including                         The solution consists of a CHARIOT platform that integrates
thousands of sensors and actuators of varying types,                         the various components and services of the solution integrated
functionalities and levels of sophistications deployed across six            into a cohesive and dynamic approach. The main components
main buildings. These endpoints constantly monitor and report                consisting the CHARIOT solution include three run-time
back to different systems such as safety and workplace                       engines: i) privacy engine ii) security engine and iii) safety
management systems. The endpoints range from state-of-the-art                engine, each responsible for different layer of IoT data
fire detection sensors down to inexpensive heat sensors placed               management and security. Machine Learning (ML) technologies
in computer racks in internal lab rooms by operations staff.                 are running in both the safety and privacy engines to ensure that
These systems perform monitoring and control functions in an                 data are inside the predictive boundaries and follow normal (and
isolated manner. Each system is an IoT silo that has visibility              acceptable) operational behaviors inside the networks.
over a limited area and has actionability to perform a constrained               The solution also integrates recent research results on
set of functionalities only. In addition, these heterogeneous                software level guarantees, including source code analysis
systems contain different user interfaces, which makes it                    (development time) and binary code analysis (execution time).
difficult for administrators to get used to and use them to their            These are strongly interconnected (via metadata interchanges
full potential. This makes the enforcement of campus wide                    into the security engine) to provide an end-to-end IoT devices
safety and security policies extremely difficult to realise. In fact,        lifecycle management and security at the firmware level.
in the best of cases, these systems only allow for basic analysis
of aggregated and historical data collected through some                         A strong component of the solution includes a blockchain
datapoints spread across multiple silos on the campus.                       layer combining Public Key Infrastructure (PKI) technologies to
Visualization and reporting of intrusions, out of boundary                   affirm firmware or devices modifications storing the related
behaviour as well as end to end devices lifetime monitoring                  information in a Distributed Ledger approach. This is used for
(software upgrades etc.) are of primal importance and need.                  both the devices’ network registration (and commissioning) and
                                                                             also for the firmware updates (guarantees of IoT device
C. Airport Environment Requirements and Challenges                           firmware) from source code development up to the firmware
    In airport situations, as analyzed from the Athens                       update at the device. Operational and management dashboards
International Airport (partner in CHARIOT), the primal                       serve as the User Interface (UI) for the platform and system
importance of the operators is focusing on evacuation cases,                 operators including IoT sensors/devices commissioning,
passengers’ comfort and maintaining smooth conditions in both                network setup, management and control as well as zones’
cases. For this, monitoring/sensing systems are spread in various            definition and topology considerations.
places of the airport infrastructure and continuously monitor the
infrastructure sensor measurement to ensure in bounds                            As described above, a reference architecture integrates all
behaviour. However, tampering (software or hardware) of these                above modules and technologies into a modern IoT solution
devices remains practically impossible (or very difficult), airport          span inside the cloud and fog layer of services. A high-level
operators remain seriously alert in keeping up with modern IoT               system description is included in the diagram below:
cyber security solutions and standards to avoid this. For this,
recent cyber security implementations ensuring the data safety,
security and privacy are of outmost importance in view of
trusting the sensor data itself.
D. Train/Rail Environment Requirements and Challenges
    Cooperation with TRENITALIA (as also a partner in
CHARIOT), has revealed a different dimension also related to
data security and privacy that relates to data collection for safety
and predictive maintenance operations as well as efficiency
management. This is seen usually in train (wagon) scenarios




                                                                        12
                                                                                          ▪ Registration of sensor status and
                                                                                            alerts in blockchain affirming
                                                                                            transactions and events
                                                                                          ▪ Private data automated flagging
                                                                                            and reporting
                                                                                          ▪ Safety engine managing
                                                                                            topology, sensors deployment,
                                                                                            commissioning and provisioning
                                                                                          ▪ Data encryption policies based on
                                                                                            blockchain technologies to avoid
                                                                                            privacy breaches in IoT
                                                                                          ▪ Dashboard-based solutions for
Fig. 1. High Level CHARIOT System Design
                                                                                            sensor configuration,
                                                                                            management and alerting
    More details for the operation and capabilities of the          ▪   Unintentional     ▪ Orchestrating mechanism for
developed modules are described in the following sections in            configuration       sensor data ingestion,
this publication.                                                       changes             management, storage,
                                                                    ▪   Damages by          normalization and external
   The table below summarizes the technical orientation of
                                                                        third parties       connectivity API
CHARIOT over modern IoT threats and the particular
components of CHARIOT                                               ▪   Erroneous usage ▪ Machine learning anomaly
                                                                        by                  detection based on user-defined
 IoT Threat          CHARIOT Solution                                   administration      models and neural networks
▪ Man-in-the-       ▪ Ruggedized communication                                            ▪ IoTL (language) for dynamic
   middle attack      protocol and encrypted                                                network configuration, access
▪ IoT      protocol   communications between devices                                        control rules and network
   high jacking       and controllers/gateways                                              topology definition
▪ Network             supported by blockchain                                             ▪ Dashboard-based solutions for
   reconnaissance ▪ Provisioning of all sensors in an                                       sensor configuration,
                      IoT network through blockchain                                        management and alerting
                      registration/affirmation                      ▪   Failure of sensor ▪ Machine learning anomaly
                    ▪ Blockchain-based PKI for sensor                   or device           detection based on user-defined
                      and gateway authentication                    ▪   Software            models and neural networks
                    ▪ Four-eye-principle based sensor                   vulnerabilities   ▪ Predictive analytics to highlight
                      provisioning in the IoT network                   exploitation        out-of-bounds behaviors and
                    ▪ Dashboard-based solutions for                 ▪   Failure/malfunc     assess combined interdependent
                      sensor configuration,                             tion of control     risks
                      management and alerting                           system
▪ Malware           ▪ Firmware static analysis avoiding             ▪   Contractual       ▪ Machine learning anomaly
▪ Denial of service   software vulnerabilities (etc.) at                requirements        detection based on user-defined
▪ Software/hardw      source code and existence of                  ▪   Violation of        models and neural networks
   are/               backdoors, software scope                         rules             ▪ Predictive analytics to highlight
   info               alteration etc.                                                       out-of-bounds behaviors and
   manipulation     ▪ Firmware binary checking                                              assess combined interdependent
▪ Targeted attacks    against injected code at execution                                    risks
▪ Abuse of            level avoiding Ransomware,                    ▪   Sabotage /        ▪ Out of CHARIOT scope for
   personal data      viruses, Trojan horses and                        Vandalism           CHARIOT however support for
▪ Brute force         spyware                                                               malfunctioning devices is
                    ▪ Firmware hashing and meta data                                        provided
                      storage inside the binary (and
                      blockchain) for increased                                   IV.   THE CHARIOT IOT ENGINES
                      software update assertion                         CHARIOT integrates three (3) IoT data management layers
                    ▪ Orchestrating mechanism for                   responsible for performing operations on the data to verify and
                      sensor data ingestion,                        affirm their privacy, security and safety inside the IoT network.
                      management, storage,                          The components have been designed by taking into
                      normalization and external API                consideration the operation and scalability requirements of the
                                                                    three living labs participating in CHARIOT (rail, airport, smart
                                                                    buildings) into the IPSE (Integrated Privacy and Safety Engine).
                                                                    Safety here refers to Machine learning anomaly detection based




                                                               13
on user-defined models and neural networks. The IPSE can be                     To build the Privacy Engine, open source solutions and
scaled out by distributing the runtime across multiple nodes if             Python scripts have been used to develop this application. For
needed. A CHARIOT simulation tool will also be used                         encryption an RSA algorithm was used to complete the engine.
internally to test the platform and overall system scalability and          The solution was packed as a docker container and it is available
elasticity through exhaustive testing using large series of data            at GitLab Private Registry.
that may not be available in the CHARIOT LLs but still pose a
significant challenge in IIoT systems and networks. These are               B. Security Engine
described below:                                                                The CHARIOT security engine is responsible for the
                                                                            integrity and trust of the devices (sensors, gateways, controllers
A. Privacy Engine                                                           etc.) of the IoT network. This protects the devices (and network)
    The CHARIOT Privacy Engine employs and integrates                       against modern IoT attacks such as: i) reverse-engineer of the
modern security protocols and technologies (e.g. Blockchain) to             entire firmware (extract the file system and understand how the
provide the foundation layer for the trusted interchange of                 entire device works, knowing the possible use of known-to-be-
information between the different network actors (sensors,                  vulnerable out-of-date API/libraries or unknown exploitable
nodes, devices, gateways, controllers etc.). The Privacy engine             vulnerabilities), ii) insert a firmware backdoor (making the
utilizes the IoT topology described with the IoTL language to               device covertly connected to a malicious Command & Control
ensure that only data from well-known sensors are accepted into             server), iii) change the device behaviour (altering its
the system. The IoTL language itself was extended with new                  performance), iv) find hard-coded private symmetric-
concepts that can fully describe access control rules and allow             cryptography keys/passwords/user-names or private certificates
access to sensor data only to specific systems, users, roles, etc.          (used to encrypt communications between the device and other
These new concepts also add semantics relevant to privacy, such             systems and eavesdrop these communications) and v) roll-back
as explicitly flagging a sensor as a sensitive data sensor, that can        the firmware to a previous legitimate version with known
later be used e.g. to obfuscate or anonymize some or all                    vulnerabilities he/she wants to exploit (verify if the pushed
properties of the data [4]. When a system needs to receive sensor           firmware is authentic, so it can easily survive most of the in-
data it must register its public key with CHARIOT’s                         place controls, as usually, they tend to check just the firmware
Blockchain-based PKI. The Privacy engine uses the PKI to get                source and/or the firmware integrity) [6].
the public keys of the system that is allowed to receive sensor
data and uses it to encrypt the data before sending them. This                   The CHARIOT security engine verifies the reliability of new
way only the owner of the private key can decrypt and access                issued firmware(s) during the tricky and demanding update
the raw data [4].                                                           phase using features detection and heuristic approach. The
                                                                            firmware verification analyses the firmware’s binary that will be
    This component considers recent privacy issues in IoT                   flashed on the end-device (sensor or gateway). The firmware
systems including data being collected by individual sensors that           analysis is performed during the firmware update process, and
should enter the system if only the sensor is known and                     its purpose is to highlight any vulnerabilities inside the firmware
registered in the topology and also if the data is from a known             code that could potentially lead to cyber-attacks. A created hash
sensor, data encryption must be applied using a public key stored           (during the firmware development stage) of the firmware is
in a blockchain PKI. This module uses advanced cryptography                 stored in the blockchain after the validation of the Security
in achieving protection towards confidential information stored             Engine. The hashing of the binary file is performed by the
in network and secure transmission over one network to another              CHARIOT platform along with the keypair and the registration
network. Cryptography is applied on the sensor data,                        of the hashing to the blockchain. When a potential security issue
immediately after, sensor data are verified over their receival             has been found inside the reversed binary code of the firmware,
from a (topology) well known sensor. CHARIOT has designed                   the Engine reports a security violation to the management for the
the encryption PKI engine so it can support multiple encryption             subsequent actions and analysis.
algorithms and has initially adopted the RSA Cryptography
algorithm for the first version of the Engine. The integrated                   The heuristic method treats the system as different sub-
blockchain layer provides valuable security features such as                systems so that the sub-system's solution must spread widely at
certificate revocation, elimination of central points-of-failure            the solution space. This approach is more appropriate since we
and a reliable transaction record that are otherwise unattainable           have to deal with types of firmwares that are often very different
by traditional PKI systems. Additionally, blockchain is applied             from each other (in architectures/CPUs/ characteristics).
as a public append-only log, naturally provides the certificate             Heuristic method brings several benefits, giving us flexibility in
transparency (CT) property proposed by Google [5].                          analysis, in fact we can combine different features as well as
                                                                            news instructions and features could be added as new functions
    The CHARIOT Privacy engine ensures data privacy through                 with new parameters for analysis. This allows an analysis
encrypting data at the source, specifically at the southbound               addressed by considering different aspects of the characteristics
dispatcher through a PKI supported by CHARIOT blockchain                    of the firmware, the change of its behavior and possible
infrastructure. Using CHARIOT Blockchain solution for                       vulnerabilities that could be exploited to tamper the firmware,
handling PKI provides secure encryption for the multiple data               leading to a more complete and reliable analysis.
streams handled by CHARIOT. Alert flags are raised in every
case of sensitive data transfer through the fog-node; thus, the                 The utility is designed to collect data by binaries, perform
Network Administrator is informed in order to report                        statistical analysis, compare two firmware images and checking
accordingly.                                                                for vulnerabilities and formal contracts. The analysis is
                                                                            performed on the assembler instructions level. Based on the




                                                                       14
analysis results, a report is generated which contains information        interactions with the service topology as well as static and
on the differences between the two images and if a vulnerability          dynamic policies enforcement. The IoT Manager UI is
has been detected. An advanced attack pattern recognition helps           implemented using the React and Leaflet libraries and features
to detect unusual hardware behavior and compares anomalies                a Quake-style terminal for inputting IoTL commands [8].
with an internal set of instruction that can lead to recognize an
unknow attacks and exploitations [6].                                           V.    PREDICTIVE MACHINE LEARNING MODELLING
                                                                              IoT data are in general characterized by volume, velocity and
                                                                          variety-lack of structure/heterogeneity. The frequent lack of
                                                                          structure in IoT data makes it difficult to analyze such data with
                                                                          traditional analytics and business intelligence tools.
                                                                          Additionally, IoT data that capture physical processes such as
                                                                          temperature, motion, or sound can be noisy. Finally, the quality
                                                                          of IoT data can vary, i.e. datasets can have significant gaps, and
                                                                          contain corrupted readings. Lastly, meta-data/context may be
                                                                          essential to understand IoT data, as such data are often
                                                                          meaningful in some context. IoT data typically contain patterns
                                                                          that include seasonal fluctuations and trends. Such patterns must
                                                                          be detected amongst noise, random fluctuations and other non-
                                                                          important findings. IoT analytics systems can filter, transform,
                                                                          and enrich the IoT data before storing it, usually in a time-series
                                                                          data store for analysis. Insights from the IoT analytics are then
                                                                          used to better understand the system measured by the IoT
                                                                          sensors and to make better decisions.
                                                                              Anomaly detection refers to the problem of finding patterns
                                                                          in IoT data that do not conform to some norm [9]. These non-
Fig. 2. CHARIOT Security Engine Model Implementation [6]                  conforming patterns are often referred to as anomalies, (and also
                                                                          as outliers, exceptions, aberrations, etc.) in different contexts.
    The CHARIOT security engine vulnerability detection layer
                                                                          Anomaly detection has wide applicability in a variety of IoT
provides the following vulnerability classes check: i) buffer
                                                                          applications such as for security protection and fault detection in
overflow, ii) format string and iii) artbitrary memory access and
                                                                          industrial systems. One major application of anomaly detection,
reports its findings during the firmware update process to the
                                                                          of relevant to CHARIOT is fault detection in mechanical units.
platform and in-turn to the User Interface, accepting or
                                                                          The anomaly detection techniques in this domain use IoT to
declining/stopping the firmware update process.
                                                                          monitor the performance of industrial components such as
C. Safety Engine                                                          motors, turbines, and other mechanical components to detect
    The CHARIOT Safety Engine analyses the IoT topology and               when maintenance of the system will be required (‘predictive
signal metadata relative to the relevant safety profiles and              maintenance’).
applies closed-loop machine-learning techniques to detect safety              CHARIOT is using several different methodologies for the
violations and alert conditions. This comprises a later capability        anomaly detection layer including: i) One Class Support Vector
on the cognitive engine that will leverage the Cyber-Physical             Machine (OSVM) - trained using both positive and negative
topological representation of the system-of-systems combined              examples, however studies have shown there are many valid
with the security and safety polices.                                     reasons for using only positive examples, ii) Elliptical Envelope
    Anomaly detection aids finding patterns in data that do not           (EE) - based on the Minimum Covariance Determinant (MCD)
conform to expected behavior [7]. Under IoT terms, anomalies              estimator the first affine equivariant and highly robust
are considered as any abnormal data stream pattern whose root             estimators of multivariate location and scatter and iii) Isolation
cause may have safety security implications. These may be a               Forest (IF) - efficient unsupported machine learning algorithm
faulty sensor, a safety hazard or a security issue. By identifying        for anomaly detection focusing on identifying the few different
these issues and providing a central alerting mechanism,                  points of the dataset, rather than the normal data, and uses the
CHARIOT will help operators in reducing response time and                 isolation mechanism that detects anomalies purely based on the
identify root causes in cases of issues.                                  concept of isolation without employing and distance or density
                                                                          measure, which is fundamentally different from previously
    The CHARIOT security engine uses rule-based policies with             described methods [11].
simple arithmetic comparisons to enforce policies on data
streams. An innovative IoTL (IoT scripting language -IOT
Language) supports alerting the industrial gateway if a safety
policy violation is observed within the IoT State. Furthermore,
the security engine is using machine learning based anomaly
detection.
   In addition to a low-level Swagger API, IBM has developed
a high-level UI for interfacing with the IoTL to facilitate




                                                                     15
                                                                         monitor, tentatively called bismon. It is preferable (see Free
                                                                         Software Foundation) to use free software GCC plugins (or free
                                                                         software generators for them) when compiling proprietary
                                                                         firmware with the help of these plugins; otherwise, there might
                                                                         be some licensing issues on the obtained proprietary binary
                                                                         firmware blob, if it was compiled with the help of some
                                                                         hypothetical proprietary GCC plugin.
                                                                             CHARIOT static analysis tools will leverage on the
                                                                         mainstream GCC compiler (generally used as a cross-compiler
                                                                         for IoT firmware development). Current versions of GCC are
                                                                         capable of quite surprising optimizations (internally based upon
                                                                         some sophisticated static analysis techniques and advanced
                                                                         heuristics). But to provide such clever optimizations, the GCC
                                                                         compiler has to be quite a large software, of more than 5.28
                                                                         million lines of source code (in gcc-8.2.0, measured by
                                                                         sloccount). This figure is an under-estimation, since GCC
                                                                         contains a dozen of domain specific languages and their
                                                                         transpilers to generated C++ code, which are not well
                                                                         recognized or measured by sloccount.
                                                                             Since a single Bismon process is used by a small team of IoT
Fig. 3. Example of Anomaly Detection Modelling                           developers, it provides some web interface: each IoT developer
                                                                         will interact with the persistent monitor through his/her web
          VI.    SOFTWARE LIFE-CYCLE MANAGEMENT                          browser. In addition, a static analysis expert (which could
    The CHARIOT software analysis and lifecycle management               perhaps be the very senior IoT developer of the team) will
includes a software source code verification analysis level              configure the static analysis (also through a web interface) [13].
(Bismon) that is strongly linked to the CHARIOT security
engine (and the firmware update process). This, includes the                VII. SUPPORTING BLOCKCHAIN AND PKI TECHNOLOGIES
source code analysis, creation of metadata and hashing of source             The blockchain component of CHARIOT (based on a
code inside the binary file that are analysed during the firmware        hyperledger Fabric implementation) is used at different engines
update process (via the security engine and together with the            and layers to affirm data, devices and network information. In
binary level warnings) to either accept or decline the software          this, the information stored in blockchain include sensor IDs,
update process.                                                          network states and firmware validation hashings. These are used
                                                                         by the privacy, security and safety engine as described before.
    CHARIOT focuses mainly on a system of systems (e.g.
networks of systems and systems of networks) approach, so [10]               Blockchain-based PKI approach makes MITM attacks
“aims to address how safety-critical-systems should be securely          virtually impossible as when group of authorities publishes or
and appropriately managed and integrated with a fog network              revokes the public key of an identity on the blockchain, the
made up of heterogeneous IoT devices and gateways.”. Within              information will be distributed across all nodes, so tampering the
CHARIOT, static analysis methods support its Open IoT Cloud              public-key will be (theoretically) out of the question. Traditional
Platform through its IoT Privacy, Security and Safety                    PKI resolves MITM risks by embedding Root CA certificates
Supervision Engine. Some industrial CHARIOT partners, while              into browser installations, thus artificially expanding CA
being IoT network and hardware experts, acknowledge that their           entrance barriers and increasing the time necessary for Root CA
favourite IDE (provided by their main IoT hardware vendor) is            certificate revocation.
running some GCC under the hoods during the build of their
                                                                             There are several advantages of using this PKI-based
firmware. Nevertheless, these partners do not use static source
                                                                         blockchain implementation including: i) The validation of a
code analysis tools.
                                                                         certificate is simple and fast with no form of CA certificate
    The CHARIOT approach to static source analysis leverages             chain, ii) Blockchain-based PKI solves a longstanding problem
on an existing recent GCC cross-compiler [11] so focuses on              of traditional PKIs by not requiring the use of a service that
GCC-compiled languages [12]. Hence, the IoT software                     issues certificate revocation lists (CRLs) thanks to blockchain
developer following the CHARIOT methodology would just                   synchronization between network’s nodes where any
add some additional flags to existing gcc or g++ cross-                  modification to the state of a certificate will be instantaneously
compilation commands, and needs simply to change slightly                notified to the all nodes and iii) Blockchain-based PKI provides
his/her build automation scripts (e.g. add a few lines to his            flexible protection against the man-in-the-middle (MITM)
Makefile). Such a gentle approach (see figure 1) has the                 attacks. Traditionally, MITM is considered as a major security
advantage of not disturbing much the usual developer workflow            risk implying attacker to hijack a browser’s connection for a
and habit, and addresses also the junior IoT software developer.         given website by presenting a valid certificate (i.e., forged public
The compilation and linking processes are communicating -via             key) for that domain. For users and web browsers it is difficult
some additional GCC plugins (cf. GCC Community [6] §24)                  to identify the replacement of certificate when the related CA
doing inter-process communication- with our persistent                   has been hacked by the attacker [7] [8].




                                                                    16
       VIII. OPERATIONAL AND DEVICE MANAGEMENT                             in the end the system administrator has a dashboard to view all
                     DASHBOARDS                                            the collected information [14].
    User interfacing is considered as an important layer where
two distinct interfaces (dashboards) are being developed
(Device Management Dashboard: handling blockchain devices
registration, firmware updates, engine management and IoTL
interfacing and Operational Dashboard: providing Engines’
health and performance monitoring as well as alerts’ and sensor
data visualization).
    The device management dashboard is utilizing the latest
state-of-the-art web technologies to deliver rich content
information to the LL users and achieve cross-browser and
multi-device compatibility. Further to that, the dashboard is
designed as a user friendly and fully responsive web solution,
based on the CHARIOT industrial needs, providing an easy
access to the necessary information. Blockchain security and               Fig. 5. Example of Operational Dashboard
accessing controls are applied to secure the access to specific
information and data by different users. Moreover, Dashboards                           IX.    CHARIOT INDUSTRIAL VALIDATION
focus not only to standard monitoring actions and providing a
visibility on an industrial IoT topology, sensor values and alerts             CHARIOT is by design driven by industrial IoT
but also to secured (utilizing blockchain technology) managerial           requirements following actual needs and paradigms of three
activities. Those activities such as authenticating and registering        sectors: rail, airports and smart buildings. These three industrial
(or unregistering) a sensor in the IoT topology and updating the           cases’ analysis has derived exhaustive sets of requirements,
firmware (of a sensor or a gateway) can be performed by the                industrial scenarios and validation KPIs on which, CHARIOT,
security engineers and management. It is important to mentioned            has based its technical implementations.
that during the “firmware update” there is a chain of actions and              CHARIOT will be validated in the above three (3) industrial
integration with a number of CHARIOT components.                           cases based on representative security related scenarios
                                                                           highlighting the value and integrated approach of CHARIOT in
                                                                           solving modern IoT security issues and challenges.
                                                                               CHARIOT is currently through its deployment and
                                                                           validation phase, having deployed its whole platform in the three
                                                                           infrastructures and having performed its first round of technical
                                                                           recommendations from the end-users. In the next five months,
                                                                           and up to the end of 2020, CHARIOT is expected to finish its
                                                                           activities with the final feedback of recommendations and
                                                                           adaptations to the three industrial setups.
                                                                                                      ACKNOWLEDGMENTS
                                                                               This project has received funding from the European
                                                                           Union’s Horizon 2020 research and innovation program (No
                                                                           780075). The authors acknowledge the research outcomes of
Fig. 4. Example of Data Management Dashboard                               this publication belonging to the CHARIOT consortium.

    The CHARIOT Operational Dashboard is providing                                                       REFERENCES
Engines’ health and performance monitoring as well as alerts’              [1]   K. Loupos - INTEGRATED SOLUTION FOR PRIVACY AND
and sensor data visualization. CHARIOT has identified the need                   SECURITY OF IOT DEVICES IN CRITICAL INFRASTRUCTURES,
                                                                                 Critical Infrastructure Protection and Resilience Europe (CIPRE 2020),
of a more sophisticated method for platform performance                          6-8 October 2020, Bucharest, Romania.
monitoring as designed following the micro-services software               [2]   K. Loupos, A. Papageorgiou, A. Mygiakis, B. Caglayan, B. Karakostas,
architecture paradigm. After research on the industry-standard                   T. Krousarlis, F. Vedrine, C. Skoufis, S. Christofi, G. Theofilis, H.
of micro-service platform monitoring techniques, CHARIOT                         Avgoustidis, G. Boulougouris, A. Battaglia, M. Villiani - COGNITIVE
has decided to adopt CNCF best practices and deploy Jaeger.                      PLATFORM FOR INDUSTRIAL IOT SYSTEM SECURITY, SAFETY
With Jaeger, we can trace every action trail at the CHARIOT                      AND PRIVACY, Embedded World 2020 Conference and Exhibition, 25
                                                                                 - 27 February 2020, Nuremberg, Germany.
platform. The analysis of the collected traces helps the developer
to identify bottlenecks to improve system performance and find             [3]   Adel S. Elmaghraby, Michael M. Losavio, “Cyber security challenges in
                                                                                 Smart Cities: Safety, security and privacy”, Journal of Advanced
the cause of platform malfunction. In addition to this, we                       Research Volume 5, Issue 4, pp 491–497, 07/ 2014.
implement service to monitor health of every micro-services by             [4]   CHARIOT – D3.2 – IoT Privacy Engine based on PKI and Blockchain
sending a “magic-package” to it and then wait for its response,                  technologies, CHARIOT 2019.




                                                                      17
[5]  L. Axon and M. Goldsmith, “PB-PKI: A privacy-aware blockchain based
     PKI,” in Proceedings of the 14th International Joint Conference on e-
     Business and Telecommunications (ICETE 2017) - Volume 4:
     SECRYPT, Madrid, Spain, July 24-26, 2017., 2017, pp. 311–318.
[6] CHARIOT - D3.8 – IoT Security Engine based on vulnerability checks,
     CHARIOT 2020.
[7] Chandola, Varun, Arindam Banerjee, and Vipin Kumar. "Anomaly
     detection: A survey." ACM computing surveys (CSUR) 41.3 (2009): 15.
[8] CHARIOT – D3.9 - IoT Safety Supervision Engine (ISSE) (final
     prototype) v1.0_FINAL, CHARIOT, 2020.
[9] Chandola, Varun, Arindam Banerjee, Vipin Kumar. Anomaly detection:
     a survey. ACM Computing Surveys, September 2009.
[10] Taken in October 2018 from https://www.chariotproject.eu/About,
     §Technical Approach.
[11] The actual version and the concrete configuation of GCC are important;
     we want to stick -when reasonably possible- to the latest GCC releases,
     e.g. to GCC 8 in autumn 2018. In the usual case, that GCC is a cross-
     compiler. In the rare case where the IoT system runs on an x86-64 device
     under Linux, that GCC is not a cross-, but a straight compiler.
[12] The 2019 Gnu Compiler Collection is able to compile code written in C,
     C++, Objective-C, Fortran, Ada, Go, and/or D.
[13] CHARIOT – D1.5 - Specialized Static Analysis tools for more secure and
     safer IoT software development (ver.2).
[14] CHARIOT – D6.9 – CHARIOT Rescoping Guideline




                                                                                18