The present time is the time of digital transformation. New smart or intelligent devices and technologies influence our everyday life and have a great impact on our environment. Such paradigms as Internet of Things (IoT) and Smart Systems are also unavoidable undergoing changes and modifications. Especially the domain of Smart Cities is getting more and more attention and is benefiting from the outcomes of digital transformation. In this paper we focus on the evolution of technologies in this fields and provide an overview over a selection of numerous applications.
Digital Transformation, Digitization, Internet of Things (IoT), Smart Cities, Smart Systems and Big Data are s ome of the most discussed topics in recent years . Hardly anybody today does not possess a smartphone or some smart devices. Many activit ie s and events private as well as public are digitally supported and displayed. As the critical mass of users has increased, many technologies hav e become important to business and government as well. Cities, on the other hand, are still not very advanced, except for a few exceptional projects. There is a lot of potential in the use of (smart) digital technologies, especially regarding big data.
On the other hand, digital support also involves new challenges . For example, the monitoring of the entire transport network brings immense benefits to public safety and simplifies everyday life for individuals through real-time reporting of congestion or accidents . However, concepts must also be developed to protect this data and personal information. This paper addresses the evolution of big data architecture, IoT and smart urban environment and highlights some of the challenges, but does not discuss legal implications or privacy issues.
Along with the progress of digital transformation a new dimension arises in the
way how people interact with things in their surrounding environment. This evolution
reveals a whole new area of research challenges and emerging technologies , as
cumulated by Gartner [
The objective of this paper is to give an overview on changes and modifications of some selected important concepts and technologies as an implication of the digital transformation progress.
The next section describes at first briefly some of the most commonly used frameworks for processing big data. Then, the evolution tendency of IoT and Softwaredefined Systems is presented, followed by some selected trends in the area of smart cities. 2
Big Data is one of the most discussed topics in the past and nowadays. The amount o f
produced and stored data from our environment, social networks or business sector is
increasing every day. The requirements of big data applications are often described
using the five V's: volume, variety, velocity, veracity and value [
The challenge regarding big data is not anymore to collect and store data, but to
analyze it efficiently and to extract its value. This leads to complex workflows, which
require systems built for big data processing. One option for such workflows is to use
projects from the Hadoop Ecosystem [
Apache Hadoop was initially introduced in 2007 as an open source implementation of MapReduce processing engine linked with a d istributed file system. Since then it has developed into a collection of more than 100 projects related to every step of a big data workflow, including data collection, storage, processing, and much more.
The whole ecosystem is based on HDFS (Hadoop Distributed File System) and a framework for job scheduling and cluster resource management YARN (Yet Another Resource Negotiator). In the following some data processing engines are listed that are often used to develop analytic workflows for big data.
Apache Spark applications range from finance to scientific data processing and
combine libraries for SQL, machine learning, and graphs. Spark has a similar
programming model to MapReduce but extends it with a data-sharing abstraction called
resilient distributed datasets, or RDDs. Spark is most commonly used with cluster file
systems like HDFS and key-value stores like S3 or Cassandra. It can also connect with
Apache Hive as a data catalog. Spark supports batch and stream processing [
Apache Storm is a free and open source distributed real-time computation system. Storm uses streams for real-time processing and can be used almost with any programming language.
H2O H2O is an open source framework that provides a parallel proces sing engine, analytics, math, and machine learning libraries, along with data preprocessing and evaluation tools. Additionally, H2O offers a web-based user interface, making learning tasks more accessible to users who may not have strong programming experience. The framework supports Java, R, Python, and Scala. In addition to its native processing engine, there is also Sparkling Water, which integrates Spark and Spark Streaming into H2O. H2O4GPU uses GPU computation to speed -up on model training time and inference time.
TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) commu nicating between them. TensorFlow was originally developed by researchers and engineers from the Google Brain Team for the purposes of conducting machine learning and deep neural networks research. TensorFlow is a mighty tool for analysis and processing of big data.
Apache Kafka is a distributed publish-subscribe message delivery system. It is
characterized by being highly scalable, like it can scale to a large number of data sources
simultaneously sending data at high rates, and reliable, since it is fault tolerant against
failing of computing machines (no data loss). Moreover, it is a powerful tool for
building real-time data pipelines with high throughput and low latencies [
Another advantage of Kafka is that it guarantees at -least-one delivery of data from
producers to consumers. A topic can be divided into several partitions that are located
across different brokers which leads to Kafka's capability of delivering a large volume
of messages simultaneously [
With the emergence of a new generation of cheaper and smaller wireless devices and numerous technologies including sensors, actuators, embedded and cloud computing, many things are becoming wirelessly connected to the Internet. Thus, the number and variety of devices that are used in this way has increased enormously. Such devices can vary in many aspects, e.g. in size, computing resources, operating mode, network technologies, protocols, usage scenario etc.
This concept is broadly known as the Internet of Things (IoT). In [
Use cases in the IoT vary from home safety and management system, smart electricity monitoring in electricity grids, in-car system from road traffic to health monitoring and disaster management.
The concepts of IoT Architecture are broadly discussed in literature. In our
previous research [
Software-defined Systems are emerging and promising technologies to meet the
needs of current IoT requirements. Software-defined networks (SDN) provide a
centralized control and global view of the whole managed network by decoupling the
control functionality from the forwarding plane and program network service located
above the control plane. The centralized management allows efficient and automated
optimization as well as configuration of the managed network [
In legacy networks the control and data plane are bundled inside the networking device, which reduces flexibility because devices need to be individually configured and control functions are often hard-coded. Thus, impeding innovation and evolution of the network infrastructure. Any changes to the network infrastructure are extremely expensive in terms of cost and time.
The software-defined network paradigm is considered to be an alternative to
overcome the existing architectural challenges by decoupling the control plane from the
forwarding plane in a three-layer architecture. However, the improved flexibility in
network configuration and resource management that is provided by the higher level
of abstraction can come at the cost of performance deterioration [
The three-layer architecture, as presented in [
The separation of concerns introduced between the definition of network policies, their implementation in forwarding devices, and the forwarding of traffic is an important consequence of the SDN paradigm. This separation is the key property for the desired flexibility, breaking the network control problem into manageable pieces. Furthermore, it allows to easier create and introduce new abstractions in networking, simplifying network management and facilitating network evolution and innovation.
The immense increase in IoT devices inexorable results in huge amounts of data that
is generated, transported, collected and analyzed. As a result of this development
more and more storage systems are deployed. Virtualization techniques are used to
utilize the underlying hardware to its fullest potential and support building
manageable storage systems [
However, the complexity of a layered approach leads to the difficulty to enforce
end-to-end Quality of Service policies, e.g., bandwidth, latency or priority
requirements. These deficiencies make it necessary to establish a software -defined design of
storage system, i.e., Software-Defined Storage (SDStore) system. The SDStore
approach is based on the SDN paradigm and makes use of a logically centralized
controller that has the ability to implement different policies on different kinds of flows
[
Traditional security mechanisms are not suitable for the SDSys architectures and
therefore require new security mechanisms. Like all SDSys, Software-Defined
Security (SDSec) provides a way to design, deploy and manage security mechanisms by
separating the forwarding and processing plane from the security control plane. Thus,
SDSec abstracts the security policies, e.g. intrusion detection, firewalling and others,
from the hardware layer and run it in an independent software layer. The separation
provides a scalable distributed security solution, which virtualizes the security
functions but remains manageable as a single logical system, as [
The growing number of people living in a city poses a new set of technical challenges. This rapid progress, also referred to as urbanization, has led to many big cities producing continuous big data. The concept of a smart city was developed in order to handle all the information and to gain knowledge about a city that can help tackle the challenges.
For instance, the analysis of the city-wide human and vehicle mobility data helps to
detect problems in a city’s road network. This discovery can help better formulate city
planning for the future [
Smart mobility is often described as visionary idea of transportation [
However, in the context of smart cities, smart mobility involves not only transportation, such as autonomous vehicles, but also support for disabled or older persons as well as a real-time traffic control system.
Such a system, also referred to as intelligent transportation system (ITS), is the
application of sensing, analysis, control and communications technologies to ground
transportation in order to improve safety, mobility and efficiency. It estimates the
traffic parameter and optimizes traffic signal to reduce vehicle delays and stops by the
uses of different techniques [
The data processing approaches in ITS rely often on Apache Kafka and Apache Spark's extension Spark Streaming we briefly presented in Section 2. The traffic management centers are responsible for data analysis that is mostly done with machine learning and deep learning applications.
To ongoing digitization has already transformed the healthcare system. The concept
of electronic health (e-health) [
The development and increased use of IoT significantly transformed the health
sector [
By the transition of coal and nuclear energy towards renewable energy sources, new
concepts for the long-term storage and efficient distribution of energy are essential.
For example, in Germany [
Some of the examples of the new applications in this area are smart grids and smart meters. Smart grids represent an energy supply system that involves power generation, transmission, storage, distribution up to consumption of the electricity. The idea behind this concept is that each device which is connected to the electricity grid should be integrated on a "plug & play" basis . Smart meters are parts of a smart home and should replace conventional electricity meters . They will be integrated in the smart grids and should be able not only to measure the power consumption, but also to record voltage failures and to provide the grid operators with important information so that the power generation, grid loads and consumption can largely be adjusted automatically. 5
Digital transformation is increasingly becoming an overarching, everyday experience. It is changing the way people live, work and communicate with each other and with their surroundings. New possibilities, products and services are emerging. In order to maintain an active control of the digitization, it is necessary to monitor the changes in our environment and to adapt applied technologies making them smarter.
In this paper we presented an overview on changes and modifications of some selected concepts and technologies as an implication of the digital transformation progress. This work covers only a small part of the huge research and application area.