The design and deployment of combined Internet-of-Things (IoT) and Artificial Intelligence (AI) solutions present several challenges, some of which are handled by modular logical models for the flow of data and containerized architectures that provide virtualized environments to encapsulate diferent functionalities in a network. The combinations of containers result in reusable and general solutions, even leading to visual composition solutions with recent deployment platforms. General requirements for these solutions are the capability of handling data acquisition, maintenance, and model training from zero information. This paper studies the deployment process for a use case of visual person counting from cameras located in outdoor areas. We show how a containerized solution fulfills the particular requirements for the use case, illustrating how the design of the modular architecture, data pipelines, and exposed services contribute to enhancing adaptive behavior through learning based on the context of the environment.
This paper studies the deployment process of a use case for person counting in outdoor areas
from camera images. We describe the prerequisites that a deployment architecture must support
for this case, which include reusability, learning, and forecasting functionalities. Then, we show
our deployment solution, which follows a containerized standard and follows the guidelines for
IoT/AI cognitive architectures. This case can be tied to other field applications, such as wildlife
monitoring [
The application in our case study is based on person counting on outdoor areas to estimate the busyness in the observed areas. For the estimation, images from cameras deployed at several locations in Norway are publicly available in real time1 and are updated at a fixed interval of 10 minutes, from which we choose a subset of locations around Trondheim. The learning process is intended to be fully autonomous and adapted to the local and changing environment, therefore, no data is available before deployment. From acquired data, statistics and predictions can either be used in a closed loop to improve the behavior of constrained-resourced devices or conveyed to external users such as people planning to go outdoors and who can use a prediction of movement to choose better when to go, or those responsible for the maintenance of tracks who can better plan their work.
We focus on translating the logical data flow, summarized in Figure 1, to a solution that can be deployed. For that, we identify the following requirements that the solution should fulfill and will then describe an implementation that follows them.
• R1 The solution should be reusable and flexible to changes in its components, allowing smooth updates of the global solution and transferability to similar scenarios. • R2 The architecture should allow learning spatial and/or temporal attention maps such that statistics for these trends can be provided based on historical data. • R3 The architecture should allow forecasting models to estimate future observations, such that it can support devices by planning their resources or providing information to external users.
This section describes how we handle each requirement with a containerized solution. 1http://metnet.no/#kart_header (accessed on 30 May 2023)
Implementing an architecture for this use case follows a modular, containerized-based structure
[
Resource-constrained IoT devices can select to drop the transmission of parts of an image as a mean to save energy, which is especially beneficial when areas that never contain any persons are in the field of view of a camera (e.g., in Figure 1 part of the image captures the sky). Each device manager container implements and maintains a visual attention map that can, by request, be sent to a device as the base for a tile selection policy.
The server receives the transmitted tiles, reassembles them with a background image received
earlier, and then performs the object detection on these reduced tile sets. The details of the
algorithms for the computation of the attention models and the policies are explained in [
Figure 3 shows an example of a spatial attention map for a camera, represented as a heatmap. Whenever persons are detected in a newly received image, the heatmap is increased in the respective image slices where bounding boxes are located. Cells in the heatmap with darker coloring correspond to areas where the detection of persons is more likely. Similar attention maps can be made for temporal maps, showing the periods during each day with more detections.
This architecture includes a forecast module, which implements the functionalities for the
forecasting procedure. It exposes services for initialization, model training, and prediction. At
initialization, it receives configuration data such as the number of weeks included in the training
data. Then, whenever the manager calls for model training or requests a new prediction, the
forecast module receives the current timestamp, constructs the needed datasets with calls to
the database, and updates or runs predictions, respectively, with the stored trained model.
3.3.1. Data
To evaluate the method, we collected a dataset of detections for eight cameras between December
2021 and January 2023, some of which are co-located but with non-overlapping fields of view,
using the You only look once approach (YOLO) [
Camera jervskogen1 jervskogen2 nilsbyen2 nilsbyen3
skistua ronningen1 ronningen2 meraker1
We present a deployment solution for a use case of visual person counting from cameras in outdoor areas, combining IoT devices and AI components. The implemented solution is a reusable containerized architecture, allowing the inclusion of more devices after deployment. Using a manager component allows centralized management of the data flow and keeps updated attention maps. The separation of diferent functionalities in each container allows to add services to the system, including a service to forecast observed phenomena.
This implementation can contribute to the eficient operation of resource-constrained devices. Our use case includes energy management for devices with a limited energy budget, which can benefit from predictions on the busyness of the observed area for a more eficient use of energy towards periods when more persons are expected.
This work has been partly funded by the SFI NorwAI, (Centre for Research-based Innovation, 309834) and by the European Union’s Horizon 2020 research and innovation project AI4EU, grant agreement No 825619.