The article researches and develops a system for secure data exchange in IoT. The architecture of the secure remote update protocol and the types of messages supported by this protocol have been developed, and look at how the protocol can be used with the MQTT publish/subscribe architecture. Issues related to identity management for IoT devices are discussed, as well as an automated system identity management scheme to be used with the protocol. An experimental evaluation of the protocol and system and further analysis of the obtained data was carried out. All basic elements of the protocol are readymade components. It was built on top of the very widely used MQTT protocol. It uses TLS to encrypt messages and uses RSA and SHA-256 to sign messages in the reference implementation. This use of well-known components lends some credibility to the protocol and makes it easier for users to understand.
IoT security is a technology segment focused
on protecting connected devices and networks in
the Internet of Things (IoT). IoT involves the
addition of Internet connectivity to a system of
interconnected computing devices, mechanical
and digital machines, objects, animals, and/or
people [
Several high-profile incidents where a
common IoT device was used to infiltrate and
attack a larger network have drawn attention to the
need for IoT security [
A command and control system (C2), although
traditionally considered in a military context,
simply means the process by which a
hierarchically superior entity sets tasks for
(command) or gives direct instructions (control)
to a subordinate entity [
In the context of this study, C2 can be seen as
a reference to any communication that occurs in a
hierarchical structure where information is
exchanged between a controlling entity [
There are two main types of architectures: a purely hierarchical approach, typically used in the context of military or civilian law enforcement personnel, and a peer-to-peer approach.
In the strictly hierarchical C2 communication model, all communication between nodes must go through a (common) senior commander.
In contrast to this approach, a purely
peer-topeer communication model allows direct
communication between any nodes. An
architecture that adopts this scheme will have a
completely flat hierarchy where all nodes are
equal and any communications flow directly
between endpoints. More typically, in nominally
peer-to-peer communications (such as a telephone
system, for example), messages are routed or
relayed through a server, exchange, or other
intermediaries at one or more levels of
hierarchical abstraction from the user [
This paper primarily considers a use case
scenario commonly encountered in IoT—where a
certain class (or family of classes) of devices is
deployed and C2 functionality provided by a
centralized service [
It is also compatible with the new trend of
using IoT devices and integrating with existing
cloud services such as those provided by Google
or Amazon. This approach enables easy
integration with home automation hub devices
such as AmazonEcho (and Alexa voice assistant)
and GoogleHome [
Given the requirements for IoT and IoT systems in a military context, there is a clear requirement for a C2 system to integrate them into existing military decision-making processes and structures.
The infrastructure requirements underlying the
network connectivity for deploying IoT devices
can be met by deployed military systems or local
theater systems (or a combination of both).
However, depending on the scale of any such
deployment of IoT devices, the infrastructure may
not be sufficient to meet the connectivity
requirements to support a real-time system [
Finally, it should be noted that the technical challenges of messaging in C2 systems differ from the organizational challenges of decision-making, information management, and situational awareness that future IoT systems will also face.
As an example, consider the case where there
is a deployed IoT system consisting of a fixed
number of sensor devices, and if the devices
belong to a C2 (Command and Control) control
system, they all work as part of a smart city. In the
event of an emergency, these devices can be
supplemented by pairing them with a second set
of mobile devices operated by local fire and
rescue services. The protocol supports the
dynamic addition of devices to the C2 network, as
well as commands that require devices to transfer
their registration to another C2 server if the
corresponding systems work on the same network
[
For this paper, we consider the use of a network of fixed sensors installed to measure air quality and provide real-time alerts to citizens if air quality deteriorates below a certain threshold.
In the event of a major fire or other disasters, it may be desirable to supplement this system with several additional mobile sensors, such as those that can be deployed by the local fire department. As part of the development, this could easily be achieved if the fire brigade sensors are commanded to join the city’s C2 network. This combined system C2 is shown in Fig. 1.
More generally, for this to be possible if all of the following three conditions are true: 1. Systems must use (or be compatible and able to use) the same protocol universe. 2. The owner of the sensors to be connected to the existing C2 system agrees to (temporarily) transfer ownership of the sensors to the operator of the fixed system. 3. The owner of the fixed system accepts the new sensors and assumes temporary responsibility for them.
Conditions 2 and 3 are particularly important.
Since the devices in question will become full
members of the new C2 network, both parties
must be prepared to join the systems, which
involves a certain degree of trust. For example, the
party hosting the C2 network must be confident
that the devices that join are not compromised by
malware, and the party providing the devices must
be confident that the other party will make every
effort to ensure that they are not and that they will
be returned to their control after the operation is
completed [
However, there are scenarios where such unification cannot be achieved because one or more of the preconditions are not met and cannot be satisfied (for example, when one party does not wish to assign its devices to the control of the C2 network operator—or if the devices in question necessarily use different server systems).
However, a combination of C2 systems is not always necessary. Depending on the specific requirements, it is also possible to simply provide third-party access to the existing C2 system or provide a simple data export (in the form of static data or the form of a live API).
As part of this study, an implementation of the
C2 web system was created. This was
implemented in Python because the protocol
library [
Setting up a C2 server requires some additional steps compared to setting up a device, not least when it comes to setting up the C2 server’s security credentials. Unlike devices (which can request registration and generate credentials simply by visiting a key exchange URL), the registration process for servers requires the creation of a server token file. This, along with the server ID, must be specified in the manually generated configuration file. The C2 server reference implementation also supports humanand machine-driven joins. It allows you to use color or monochrome icons, word lists, and hexadecimal notation. It also includes a simple example of visualizing data from a device by plotting a graph, although in a real (production) system it would be highly desirable to use scalable, off-the-shelf time series data storage and visualization tools such as InfluxDb and Grafana.
It should be noted that the implemented C2 system does not use any type of client-side user authentication. However, implementing user authentication for web applications is a solved problem, and there are several ready-made OSS solutions compatible with Python implementations (such as Flask-Login7) that can be easily added for any real-world use.
A reference implementation was also created for the systems server services. It consists of two main parts: a key exchange server; and supporting infrastructure for hosting and brokering MQTT messages.
The key exchange server was implemented as
a relatively simple Python program. It implements
a REST API endpoint for all steps of the device
registration process and again uses the Flask
library to provide a web services framework [
Table 1 shows the implemented REST endpoints.
Although the key exchange server is relatively
simple (especially when running locally), the
configuration required to host it securely on a
remote web server [
The basic orchestration configuration is
specified using the YAML (YAML Ain’t Markup
Language) format and is contained in the
dockercompose.yml file [
Of these, only the KeyEx service is an individual container. The other three all use existing container images and augment them with custom configuration settings specified in the docker build file. They are presented in containers through the file system using Docker Volumes.
The Dockerfile that defines the build process for a KeyEx image simply takes the latest Python 3 container image and installs the Python library dependencies for KeyEx. These include Flask, a Cryptography 8 library, and Green Unicorn9, which implements Python.
Web Server Gateway Interface (WSGI). The
last step is to define a startup script for the KeyEx
service [
Service description Implements nginx as a reverse proxy web server Implements the key service Implements Eclipse Mosquitto MQTT broker Implements the Let’s Encrypt certification robot to generate a TLS certificate
Thus, the entire backend can be distributed as
a series of small source code files that can be
assembled using Docker and Docker-Compose to
deploy the complete SRUP universe on any server
with minimal developer intervention. Although
Docker-Compose was adopted as a reference
implementation, alternative orchestration layers
such as Kubernetes can be substituted, especially
for running large-scale deployments under high or
variable loads [
An architectural diagram showing how all the components fit together is shown in Fig. 3.
Using dynamic device identification requires solving the problem of mapping a particular physical device to whatever logical device identification it has at any point in time. In particular, this is required when this device is connected to the network of the C2 system. If a device’s identity can be established at the time it joins, it can be maintained throughout its participation in the C2 system network.
The simplest form of connection operation is a simple or unmoderated connection. It does not attempt to verify the ID of the connecting device and only accepts the request based on the credentials provided (such as the device type) during the initial device registration.
This type of association is only suitable for
situations where either there is no risk of a
malicious device being added (for example, on a
closed network within a secure perimeter) or
where the overall system is not adversely affected
by erroneous or malicious data being sent from the
device. Given the MQTT addressing model [
To combine the best elements of a binary implementation with the ease of use of a scripting language, a hybrid approach was taken to implement the SRUP software library. For this study, the binary code was written in C++, and Python was selected as the scripting language.
The design concept was to create a basic
implementation to generate and process a stream
of bytes to be used as the payload of an MQTT
message [
A Python wrapper (ruSRUPLib) [
An architectural diagram illustrating this approach can be seen in Fig. 4.
In particular, the following elements were implemented as part of this work: 1. C++ library (libSRUP_Lib). 2. Python binary library (ruSRUPLib). 3. Pure Python shell class (ruSRUP). 4. Key exchange server. 5. C2 web system. 6. Container backend.
7. Initial key generation tool.
The final piece needed to ensure the protocol is created from scratch is a means to generate the set of required certificates and keys. A total of eight of the following files are required for a system running on a network connected to the Internet: • CA certificate of the broker. • A pair of public and private keys of the C2 server (used to identify the C2 server). • C2 server token file. • Pair of public and private keys of the key exchange server (used to authenticate the KeyEx server). • C2 broker access key and certificate (used for C2 server access to the MQTT broker).
A system running on a private network (or a network not connected to the Internet) also requires two additional files: the private root key of web CA; the private web CA root certificate.
The process of generating these files, especially CA root files, is quite complex. So a key generation tool (also written in Python) was created to be shared with other backend elements.
This tool is designed to generate the initial bootstrap configuration for a new base system installation and to generate the necessary credentials (including a server token file) for any C2 servers to be used on that system.
Fig. 5 shows the system at work and the relationships between the various files created.
Using Docker, the container approach creates
the backend as a component that the user can
easily deploy without having to implement the
complexities of securing the broker and managing
the key exchange itself [
Thus, this approach allows the deployment of
the SRUP universe to be considered as a
commodity service to individuals or organizations
that have a particular requirement for the privacy
or security of their IoT systems [
Experiments are performed and described in this section to compare the relative processing time, message size, and power consumption of a system using the protocol with an identical device that uses an insecure approach and uses regular MQTT messages directly. of 3.1.
An initial evaluation of the protocol’s
performance was performed by determining the
execution time of the cryptographic functions
used to support the protocol. A test case was
evaluated by measuring the time required to sign
and verify a representative update initiation
message. This process was run five times on a
Raspberry Pi 3 B with the average time calculated.
According to the standard implementation [
The experimental system consisted of five IoT devices, each built on a Raspberry Pi 3B+ singleboard computer equipped with a special board with two LED status indicators and a button for user interaction.
The devices were connected via Ethernet to a Raspberry Pi 3B+, which acted as the C2 server. The C2 server ran custom software that randomly selected one of five devices, sent a message to that device asking it to change its LED state, and then waited for a random interval before returning. The program continued until each device received 250 messages. This workflow is shown in Fig. 6. Two separate implementations of the C2 system software were written. One uses protocol action messages over a TLS-encrypted MQTT connection, and the other uses a plain text MQTT message.
The previous assumption was that a significant proportion of any additional delay would be due to the time spent processing the cryptographic algorithms used to sign the messages. To assess the extent to which the performance of the protocol depends on hardware speed, an additional device was created using a faster Raspberry Pi 4 single-board computer, allowing performance to be compared with other devices. The Pi 4 was identical to the devices described above except for the CPU and RAM configuration changes between the Pi 3 and Pi 4.
When using the protocol in the system, an increase in the time needed to process messages was expected, as well as an increase in the power consumption of system devices (due to additional processing requirements for running message signature algorithms). The total size of the data sent had to be increased as well (due to the additional fields used by SRUP to ensure message security).
When testing the system, the following measurements were made: 1. Time. The actual performance measurement was evaluated by analyzing the log files generated by the devices and the server for each experimental run. 2. Power. The average power consumption of one of the devices during MQTT and SRUP conditions was evaluated. The measurement was made using a USB power meter. 3. Message size. Network traffic was captured using Wireshark and examined to determine the size of raw MQTT messages and the MQTT implementation of the system protocol.
All log file analysis was done using Python and Jupyter. The panda’s library was used to map and analyze the data.
This analysis included: 1. Mapping the device ID to the logical device number. 2. Download log files from the C2 server for each experiment. 3. Removing unused columns from the resulting data frame. 4. Downloading all log files from each device for each experiment and concatenating them into a single Python object (a list of data frame dictionaries). 5. Create a new data frame for each line in log frame C2, recording the device number, operation type (enabled or disabled), and the timestamp when the command was sent. 6. For each row in the data frame created in step 5, extract the timestamp when that command was received by the device. 7. Calculation of the delay time between sending and receiving, in milliseconds.
Each experiment produced a graph that was used to verify the data acquisition process (Figs. 7 and 8). As expected, experiments using a delay distribution had a significantly larger standard deviation. A similar analysis process was also performed for experimental runs using MQTT.
A graph showing the average latency for each experiment is shown in Fig. 9, and a graph showing the combined averages for each device for a given experiment (for both the protocol and MQTT cases) is shown in Fig. 10. General the average processing overhead for the protocol compared to MQTT across all network conditions showed an additional 51.60 ms. This compares to a worst case of 56.13 ms excluding the effects of network latency. Although the added latency is much smaller (58.44 ms vs. 2.31 ms) in Ethernet LAN compared to a fully unprotected system, compared to a more representative scenario for a deployed IoT system (average 4G capacity): the overhead is only 53.55% of the delay MQTT (147.7 ms vs 96.17 ms = 51.53 ms).
Even in the worst case, the processing overhead means that only if the message frequency exceeds 17.82 Hz will the additional processing time be greater than the natural period of the message (equation 1).
Since a typical real-world IoT device can be expected to have an average message-to-message time of minutes, the additional processing overhead on the order of tens of milliseconds is a very small additional price for the very significant security benefits provided by the protocol. However, this result shows that the protocol in its current form may not be suitable for very timecritical applications when running on lower-spec hardware. The analysis also shows that the system’s protocol is robust even under extremely poor network conditions. Even in the worst case, all messages were correctly received within 4026 ms (Fig. 11) due to the robust nature of the underlying MQTT protocol.
In this work, a system for secure data exchange in IoT has been researched and developed. The architecture of the secure remote update protocol and the types of messages supported by this protocol have been developed, and looked at as to how the protocol can be used with the MQTT publish/subscribe architecture. Issues related to identity management for IoT devices are discussed, as well as an automated system identity management scheme to be used with the protocol.
An experimental evaluation of the protocol and system and further analysis of the obtained data was carried out. The protocol (compared to a plain, insecure MQTT implementation) was shown to have an overall message processing latency of 51.60 ms to 42.92 ms when running on Raspberry Pi hardware, and an increased power consumption of 727.6 mW during message processing.
The Secure Remote Update Protocol provides an efficient design and implementation of secure C2 messaging for IoT applications. All basic elements of the protocol are ready-made components. It was built on top of the very widely used MQTT protocol (which itself runs over TCP). It uses TLS (including AES) to encrypt messages and (in the reference implementation) uses RSA and SHA-256 to sign messages. This use of well-known components lends some credibility to the protocol and makes it easier for users to understand.
Experimental evaluation of the system and protocol has shown that the system performs successfully even under extremely poor network conditions (far beyond those typically seen in the vast majority of real-world deployments). Experiments have shown that despite the performance overhead associated with using the protocol, it is a relatively small factor for most realistic use cases.