Current mainstream approach to sensor data monitoring usually relies on cloud access: samples are acquired by connected devices and data processing is performed on remote servers. To improve responsiveness, security and resilience, devices and programming methodologies must be improved, with the aim of enabling data analytics at the edge. Unfortunately this is not an easy task, especially in the IoT domain. In this paper, we present a research approach that manages at runtime the hardware/software con guration of a low-power processing system, with the aim of adapting to dynamically changing workloads optimizing power-relevant settings to the corresponding operating point. First, we present a rst validation experiment, involving a hardware-software architecture for a connected sensor-processing node that allows the set of in-place processing tasks to be executed to be remotely controllable by an external user. The designed system is capable of dynamically adapting its operating point to the selected computational load, to minimize power consumption. The bene ts of the proposed approach are tested on a use-case involving ECG monitoring, that, when selected, performs ECG classi cation using a lightweight convolutional neural network. Experimental results show how the proposed approach can provide more than 50% power consumption reduction for common ECG activity, with less than 2% memory footprint overhead and recon guring the system in less than 1 ms. Second we present our plans to extend this approach to more complex multi-core systems.
Modern systems-of-systems (SoSs) are typically composed of multiple hierarchies of sensory nodes, that are in charge of collecting huge amounts of sensorial data from the physical environment. Near-sensor data analytics is often exploited, to convert acquired raw sensor data in more compact information, with the aim of reducing bandwidth requirements and communication-related power consumption, increasing responsiveness of the system and avoiding transmission of sensitive data for privacy reasons. A further improvement of this kind of approach is provided by recent advances in machine learning and cognitive computing techniques, such as those in the eld of deep learning and neural networks, that provide very high performance in terms of accuracy when it comes to recognition and classi cation tasks applied to the sensed data.
To combine near-sensor processing with energy e ciency, data analytics tasks, that may be quite compute-intensive and power-hungry, must be activated only when needed. Nodes must adapt, even at runtime, to di erent operating modes corresponding to di erent usage cases, each one optimized for power e ciency.
In this paper we present a methodology enabling such kind of runtime
dynamic management. A detailed description of this experiment is presented in
[
To validate our approach, we have chosen an application in the IoT health
domain as a use case. The value of such market is projected to reach $136 billion
by 2021 [
Several research activities have started exploiting current advances in
machine learning and arti cial intelligence for detecting speci c events/conditions in
sensed data. Some interesting approaches can be found in [
As main novel contribution in this work we propose: { An implementation of a remotely-controlled sensory node allowing for nearsensor data processing inserted in an IoT context. { Its validation on a state-of-the-art data analysis based on a Convolutional
Neural Network as an example computational load. { The evaluation of the e ectiveness of in-place computing and operating mode dynamic optimization, as a method to reduce the power consumption of the node, on a case study involving classi cation of ECG data. 3
The overall system architecture is shown in Figure 1. In this paper, we focus on the architecture of the sensor node. The description of the cloud infrastructure and of the gateway is beyond the scope of the work presented here.
The multi-level structure of the sensor node architecture is schematized in Figure 2. In the following sections, a detailed description of each node level is provided.
Cloud infrastructure
IoT gateway
Network of remotely-controlled
sensory nodes The SensorTile is a platform developed by STMicroelectronic, measuring 13:5 13:5mm, and equipped with a ARM Cortex-M4 32-bit low-power microcontroller (80 M Hz). Despite the small size, the device is endowed with a battery and with a Bluetooth Low Energy (BLE) module. The system exposes di erent operating modes, to be described in the next section.
Two main operating states are available, run mode and sleep mode, which can be further customized selecting the operating frequency (from 0.1 MHz to 80 MHz) and, depending on the selected frequency range, using a di erent voltage regulator.
Within our system, to dynamically reduce power consumption: { we scale operating frequency and select accordingly the optimal voltage regulator, on the basis of the actual workload to be served, { we exploit the sleep mode whenever possible. In sleep mode the microcontroller is in standby until it gets awakened by an interrupt. In our implementation the sleep mode is entered under the control of the operating system support, whenever all active software tasks are served. 3.2
We have selected a set of middleware and rmware utilities, executable on top of the hardware platform, providing basic platform management capabilities and the needed optimized processing functions. Some of these libraries are part of the SensorTile Hardware Abstraction Layer, provided by the platform vendor. The others are speci cally selected for the purpose of improving exibility and implementing cognitiveness on the sensor node.
FreeRTOS In order to enable thread-level abstraction to represent processing tasks to be executed on the platform, and to timely manage their scheduling at runtime, the FreeRTOS operating system is included in the stack. It's small enough to operate on a microcontroller (generally, size is between 4 kB and 9 kB). FreeRTOS primiteves implement real-time scheduling functionality, communication between processes, synchronization and time measurements.
In our architecture, as mentioned, FreeRTOS is able to schedule an idle task, that deactivates the system tick counter and sets the microcontroller in sleep mode. The idle task is entered whenever possible according to the scheduling queue, when no pending tasks have to be executed.
CMSIS In order to be capable of executing in-place processing of the sensed
data, we have exploited the Cortex Microcontroller Software Interface Standard
(CMSIS), a optimized library speci cally targeting Cortex-M processor cores [
Within CMSIS, we have added support for 1D convolution operators on input signals, with the aim of enabling cognitive sequence analysis. 3.3
We selected an application model based on process networks, where processes act
as data- ow actors communicating through FIFOs. Similar models are widely
used to improve analysis of concurrent tasks on embedded systems and for
studying their dynamic recon guration [
Tasks are envisioned to belong to four categories: { Get data task: takes care to collect data from the sensors. { Process task: enables in-place computing of the sensed data, e.g. to reduce bandwidth requirements and energy consumption. Multiple process tasks may be active to choose the depth of data analysis. { Threshold task: operates on the results of the processing, selecting which items are relevant to be sent to the cloud. { Send task: sends data to the cloud.
Activating/deactivating parts of the chain, the system can switch to di erent operating modes, featuring di erent trade-o s between in-place computing e ort, bandwidth requirements and monitoring precision. Examples of possible alternative operating modes will be presented within the use-case discussion in Section 4.
ADAM Along with the chains of tasks processing the di erent sensor inputs, the platform executes a periodic task speci cally dedicated to dynamic recon guration, the ADAptive runtime Manager (ADAM). This task periodically processes input messages received from the gateway and evaluates the status of the platform. On this basis, ADAM adapts the software and hardware con guration. In order to assess the bene ts of the proposed approach, we have selected a use case involving monitoring of a patient's ECG signal. In the prototype that we have implemented, an AD8232 sensor module from Analog Devices is connected to the ADC converter integrated in SensorTile, used to acquire samples to be coded using 16-bit width. Figure 4 shows components used in the prototype.
In this scenario, three possible operating modes have been identi ed (see Figure 5), described individually in the next sections. This operating mode involves complete transmission of all the collected samples to the cloud. This requires maximum bandwidth but allows the healthcare practitioner to visually analyze the whole ECG waveform. Due to the high data transmission rate, for this operating mode the system must be con gured to run at a minimum frequency of 8 M Hz. To save energy and to obtain a more stable connection, multiple samples are been grouped and inserted into a packet of 20 Bytes (8 ECG data 16 bit, 1 timestamp 32 bit). The sample rate of the ADC is set to 330 Hz, so one Bluetooth packet is sent every 24 ms. 4.2
This operating mode reduces the detail of the information that is sent to the cloud. Instead of providing visual access to the whole ECG waveform, the practitioner may monitor only heartbeat rate, prospectively only getting a noti cation in case of alert conditions. This operating mode processes samples to search for signal peaks and consequently computes the heartbeat rate.
The algorithm for detecting a peak signal is based on the its derivative. It is not vey critical in terms of time and power consumption. The measurement carried out shows that the system is capable of respecting the real-time constraints exposed by the sampling rate at a frequency of 2 M Hz. The data is sent to the cloud with a packet of 5 Bytes (1 heartbeat rate value, represented on 8 bit, 1 time-stamp 32 bit). In worst case, a packet is sent for every detected peak in the signal, thus the bandwidth requirement is dependent on the cardiac activity. Considering human tolerable heartbeat rates, the transmission rates are, however drastically reduced with respect to Operating Mode 1. 4.3
This operating mode is designed to combine bandwidth reduction with
morphological analysis of the ECG waveform. This analysis is performed in-place by
a processing task based on a convolutional neural network and communicates
occurrence of speci c patterns in the sample sequence. The implemented neural
network can recognize occurrence of anomalous events in the ECG trace, so that
communication to the cloud can take place only when a related alert has to be
noti ed. Bandwidth requirements are reduced with respect to Operating Mode
1 and may be similar to Operating Mode 2 (worst case is one packet per peak,
when all peaks are anomalous). However, with respect to Operating Mode 2,
computing e ort is higher. The node executes the 1D convolution neural
network described in [
The model can be used in three forms, using di erent size of the convolution
kernel and the size of the output (number of features) at each layer. In [
{ Conv1D large: Output size = 50, kernel size = 13.
Input size is always set to 300 samples with stride set to 1. We chose Conv1D medium as reference network, a larger network would exceed memory availability in the device. The network classi es sliding windows of input samples, centered in each ECG peak, labeling peak shapes for two possible set of categories, named NLRAV and NSVFQ (see Figure 6).
The experimental results reported in [
More details on the experimental results are reported in [
0 Heartbeat Op. mode
With frequency optimization Without frequency optimization 50 bpm 2
3 x xT tox x toN N T 100 bpm 2 3 x xT tox x toN N T 200 bpm 2 3 x xT tox x toN N T 1 e d o m . p O 6
In order to validate our approach on state-of-the-art complex parallel processing platforms and on more complex CNN structures, we will extend the presented work to perform an experiment on a multi-core platform. This would require inevitaly several improvements of what was previously described. The reference platform taken into consideration was developed within the Orlando project by ST Microelectronics. We will brie y describe the hardware components inside the system used for the sake of this project in the following. Then, we will discuss two main required extensions needed to implement runtime adaptivity on this platform, namely: { implementation of the adaptivity support on baremetal system, we will need to exploit the platform-speci c set of APIs to manage the process chain and the related operating modes. { adaptation of the parallelism level exploitable within the CNN structure, required to achieve an optimal partitioning of the workload on the available processing cores, using splitting/merging and pipeline methods. 6.1
In order to deploy these technologies in mobile and wearable devices, an e cient hardware plays a critical role for real-time operation with very limited power consumption and with embedded memory overcoming the limitations of fully programmable solutions.
We have selected as reference example a high performance and energy e cient
processor developed by ST microelectronics[
In general, we should assume (at least part of) the processing elements available in multi-core architectures to possibly not provide operating system support. Thus, when extending our work, we will need to manage the tasks in the process chain the thread-level abstractions granted by the RTOS. Moreover, we will need to consider tasks in the same chain to be prospectively executed by independent hardware cores communicating through a set/hierarchy of shared memories. The ADAM system will not use the APIs o ered by the FreeRTOS middleware.
Instead, low-level primitives provided by STmicroeletronics will allow to manage the execution and change the operating mode at runtime. Similarly to what previously shown, when a core is idle, it is placed in a low-power state. The shared memories between the cores will be used to implement software FIFOs allowing a correct communication between the di erent cores. 6.3
We will implement support for di erent kinds of parallelism, that will be exploited to optimize the partitioning of the workload on the available cores. A rst level of parallelism that we will consider will be at layer level. We will take pro t of software pipeline, assigning a core (if available) to each CNN layer. In case of signi cant pipeline bottlenecks, taking place when the execution time of a task is much longer than the others, we will allow the process network to be modi ed using split and merge transformations.
If used appropriately, splitting a layer can balance execution times of the pipeline states, allowing a more e cient utilization of the computing resources. In the example, the split transformation is performed on a Convolutional Layer block and divides it into two independent blocks. The input data is copied for every independent block. The output data stream coming from the two independent blocks is combined and synchronized by a concatenation node. The merge transformation is in the opposite direction. It composes two or more DNN graph nodes/blocks into one block and and is useful when two or more stages of the pipeline can be properly merged. We will evaluate combining design-time exploration and runtime management to nd the optimal partitioning for the selected use-cases. 7
In this work we have presented our approach to the design of sensor monitoring nodes capable of adapting at runtime to di erent workloads. We have presented a rst proof of concept, based on a hardware/software architecture for the design of dynamically recon gurable IoT, that may adapt to di erent operating modes when triggered by an external command or when suggested by the monitoring of its internal status. The proposed approach has been deployed on a commercial microcontroller platform and evaluated on a use-case involving ECG monitoring and classi cation based on state-of-the art convolutional neural network. The use-case features three di erent operating modes, selectable by means of remotely-sent recon guration messages. The results obtained show how near-sensor processing combined with dynamic optimization can be e ectively exploited, reducing power consumption up to a factor of 2 with respect to static simple monitoring, with limited overhead in terms of memory footprint and recon guration time. We have also introduced our plan for the extension of the adaptivity support to multi-core parallel computing platforms.