=Paper=
{{Paper
|id=Vol-3655/ISE2023_10_Bae_A_Ground
|storemode=property
|title=A Ground Control System Based on Digital Twin for Monitoring the Status of UAVs
|pdfUrl=https://ceur-ws.org/Vol-3655/ISE2023_10_Bae_A_Ground.pdf
|volume=Vol-3655
|authors=Bae Chang-Hui,Kim Chang-Hui,Lee Seongjin
|dblpUrl=https://dblp.org/rec/conf/apsec/BaeKL23
}}
==A Ground Control System Based on Digital Twin for Monitoring the Status of UAVs==
https://ceur-ws.org/Vol-3655/ISE2023_10_Bae_A_Ground.pdf
A Ground Control System Based on Digital Twin for
Monitoring the Status of UAVs⋆
Chang-Hui Bae1 , Chang-Hui Kim1 and Seongjin Lee1,∗
1
Dept. of AI Convergence Engineering, Gyeongsang National University Jinju, Republic of Korea
Abstract
A Ground Control System (GCS) is necessary for collision avoidance and flight path optimization
of operational Unmanned Aerial Vehicles (UAVs). However, existing GCS can only manage
parameters such as flight data and sensor data and do not offer modeling features, making it
challenging to safely monitor UAVs in rapidly changing environments. This paper proposes a
Digital twin-based Ground Control System (DGCS) for monitoring the status of UAVs. The
DGCS allows monitoring the current status of UAVs in a 3D modeled environment using digital
twin. The DGCS shows the accuracy of monitoring UAVs in the control system with an average
error rate of about 6.75cm. Also, DGCS shows the efficiency of monitoring the status of UAVs
at a rate of about 18ms and remotely controlling UAVs at a rate of about 25ms.
Keywords
UAV (Unmanned Aerial Vehicle), Digital Twin, GCS (Ground Control System), Status Moni-
toring
1. Introduction
An Unmanned Aerial Vehicle (UAV) is an aircraft that does not carry humans on board.
UAVs are recognized as key components in various fields such as reconnaissance [1],
detection [2, 3], and delivery [4]. Since UAVs fly without humans on board, they are
operated through remote control and autonomous flight. Ground Control System (GCS)
is necessary to ensure the flight management and safety of UAVs flying at a distance
from the pilot.
A GCS enables the pilot to monitor the flight path, status, and sensor data of UAVs
from the ground. Through the GCS, the pilot can understand the current status of UAVs
and, in case of danger, can remotely control UAVs. For precise status monitoring and
remote control in emergency situations, GCS can be more efficient if it is capable of
modeling environments identical to those of UAVs in flight.
Commercial GCS [5, 6, 7] for existing UAVs typically represent status and sensor data
only as parameters. Previous studies [8, 9, 10] on GCS proposed systems for controlling
and monitoring single or multiple UAVs. However, these studies have difficulty monitoring
the status of UAVs considering rapidly changing weather and surrounding environments,
making it difficult to guarantee the safety of the UAV.
ISE 2023: 2nd International Workshop on Intelligent Software Engineering, December 4, 2023, Seoul
∗
Corresponding author.
Envelope-Open chbae@gnu.ac.kr (C. Bae); kch9001@gmail.com (C. Kim); insight@gnu.ac.kr (S. Lee)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
This paper proposes a Digital twin-based Ground Control System (DGCS) for moni-
toring the status of UAVs. The DGCS can monitor the status of UAVs in a 3D-modeled
virtual environment by integrating a digital twin with the GCS. A 3D-modeled virtual
environment can provide users with the current status of a UAV in flight, reflecting
information about rapidly changing weather and surrounding environments. Through
this, the DGCS can monitor the status of UAVs more precisely with less delay time,
ensuring its safety.
To evaluate the DGCS, we measured its accuracy and efficiency. To measure accuracy,
we measured how precisely the DGCS could display the flight status of a UAV operated
in a simulation environment. As a result, the DGCS was able to monitor UAVs with
a less delay time with an average error rate of about 6.75cm. To measure efficiency,
we measured the data transmission performance in the environment where the DGCS
operates. As a result, the DGCS could monitor about 1MB of status data at a rate of
about 18ms, and it could transmit 1MB of command data to the UAV at a rate of about
25ms.
The contributions of this study are as follows: (1) It is a GCS capable of precise
monitoring. The proposed system, utilizing digital twin technology, enables precise
monitoring in a 3D-modeled environment within virtual space, considering terrain,
weather conditions, etc. (2) It is a scalable GCS. By using a digital twin, the proposed
system can add various functions such as simulation and prediction within virtual space.
2. Background
2.1. GCS (Ground Control System) for UAV
Since UAVs fly over a wide range and perform missions, a remote control and monitoring
system is essential. GCS is a system that allows controlling in-flight UAVs from the
ground.
GCS consists of hardware and software. The hardware consists of electronic devices
that establish data communication between the GCS and UAVs. The data communicated
includes flight data from UAVs and control commands from the pilot. The software
consists of the user interface. This user interface provides the current status of UAVs
and parameters for control.
GCS can interface with single and multiple UAVs. UAVs transmit their current flight
status and sensor data to the GCS. GCS can perform tracking and monitoring tasks
based on the data received from UAVs. Also, the GCS provides a feature that allows
pre-setting the flight path of UAVs. GCS allows pilots on the ground to remotely monitor
and control UAVs, enhancing safety.
2.2. Digital Twin
Digital twin was first proposed by Professor Grieves [11] from Michigan University.
Grieves explained that a digital twin consists of a physical space, a virtual space, and
the connection between the two spaces. The major feature of the digital twin is to model
the physical space identically in the virtual space to enhance interaction [12].
Digital twin allows for the real-time monitoring of actual objects existing in the
physical space within a virtual space. Also, in the virtual space, digital twin can simulate
various situations for a particular object, predicting malfunctions and performance
degradation [13, 14]. Objects in physical space can be controlled based on predicted
results.
Using digital twin allows for more precise monitoring and prediction than existing
GCS through a modeled virtual space. Furthermore, automated remote control can be
executed based on real-time UAV and surrounding environment data.
2.3. Related Work
Currently, representative GCS include QGC (QGroundControl) [5], Mission Planner [6],
and APM Planner [7]. These systems can interface with drones to monitor parameters,
such as the drone’s status data (Yaw, Pitch, Roll, etc.), and to create flight paths.
However, in these systems, it is difficult to display sensor data additionally mounted on
the UAV to the user depending on the situation, in addition to the remote measurement
data. Also, since these systems are not monitored in a 3D virtual environment, precise
monitoring is difficult.
Research is being conducted on the GCS for UAVs outside of the introduced commercial
systems. Liang et al. [8] presented a GCS for tethered UAVs. This system displays
the UAV status and mission information to the user through a GUI (Graphical User
Interface). Haque et al. [9] presented a GCS, which includes a micro-controller to allow
users to easily control the UAV. This system integrates software to display no-fly zones
and weather parameters to the user. Arco et al. [10] presented a GCS for monitoring the
missions of multiple UAVs. This system is capable of performing HIL (Hardware in the
Loop) and allows the real-time monitoring of the status of multiple UAVs through a GUI.
Table 1 shows the analysis of related work. Previous works have presented GCS for
controlling and monitoring single and multiple UAVs. However, in previous works, it
is difficult to monitor and control the status of the UAV in a rapidly changing flight
environment because they are difficult to reflect terrain and weather data. Therefore,
this paper presents a DGCS that can monitor and control UAVs in a digital-based virtual
environment by reflecting terrain and weather data.
Function
Related Work
virtualization Monitoring Control Terrain Weather
Liang et al. [8] X O O X X
Haque et al. [9] X O O X O
Arco et al. [10] X O O X X
DGCS(Our Method) Digital Twin O O O O
Table 1
Comparison of Related Works and Our Proposed Method
Figure 1: The Overview of the DGCS in UAV Environment
3. Solution
3.1. Design
This paper proposes a Digital twin-based Ground Control System (DGCS) for monitoring
of UAVs. The DGCS uses digital twin for ground control. Through the digital twin, the
DGCS enhances the interaction between the physical space (actual UAV) and the virtual
space (control system).
Fig. 1 shows the overall architecture of the DGCS. As shown in Fig. 1, the DGCS is
composed of a physical space, a digital twin (communication between the two spaces),
and a virtual space. The physical space consists of the UAV, the automatic controller
(Pixhawk), and the onboard computer for communication. The digital twin is composed
of a database to manage the UAV of the physical space and supports communication
between the physical and virtual spaces. The virtual space consists of 3D-modeled terrain,
simulation modules, and a User Interface (UI) that can interact with the user. The UI
represents DGCS, which allows users to monitor and remotely control the UAV.
The operation sequence of DGCS is as follows:
1. The onboard computer collects the status and sensor data of the UAV.
2. The onboard computer transmits the collected data to the digital twin.
3. The digital twin manages the status of single and multiple UAVs based on ID.
4. The virtual space represents the UAV in the modeled virtual environment based on
the data stored in the digital twin.
5. Users can monitor the UAV status.
3.2. Implementation
The environment to implement the DGCS used Windows 10, CPU Intel i7-11700, and
RAM 32GB. In Fig. 1, the onboard computer in the physical space used a Raspberry Pi
Figure 2: An Example of UAVs Status Monitoring through DGCS (¬ : 3D Map, : Flight Data of UAV)
4 model. The digital twin is implemented in Python, and communication between the
two spaces uses WebSockets. The virtual space is implemented as a JavaScript-based
web application, and Mapbox was used to create the 3D model.
Fig. 2 shows the DGCS, the implemented GCS. Fig. 2 ¬ shows the 3D-based terrain
and features implemented using Mapbox [15]. Fig. 2 shows the GPS (Global Positioning
System) coordinates (longitude/latitude) and altitude of the UAV currently registered in
the digital twin. In , if the current UAV is not at risk of collision, it is marked in white;
if it is passing over a building, it is marked in orange; and if it is flying at an altitude
lower than the building, it is marked in red. Also, DGCS can represent weather data
using the OpenWeather API [16].
(a) The Flight Path of UAV in Gazebo (b) The Result Monitored from DGCS
Figure 3: The Comparison of the UAV’s Movement Path in Gazebo and the Monitoring Path of DGCS
4. Evaluation
The experimental environment for evaluating the DGCS is identical to the implementation
in Section 3.2. To evaluate the accuracy of the DGCS, we experiment with whether
the DGCS can monitor the information of the operating UAV in real time. For this
experiment, we used Gazebo [17], which can simulate UAVs.
Fig. 3 shows the results of an experiment whether the DGCS can monitor a UAV
moving at about 5m/s in Gazebo. Fig. 3 (a) shows the path of the UAV in Gazebo.
Fig. 3 shows the results of the DGCS monitoring the UAV in flight within Gazebo. As a
result, we was confirmed that the DGCS can monitor the flight path of the UAV in flight
within Gazebo.
Fig. 4 shows the results of comparing how accurately the DGCS monitored the GPS
data from Fig. 3, categorized by latitude and longitude. Fig. 4a and 4b show a comparison
of latitude and longitude data from UAVs flying within Gazebo and UAVs monitored
by DGCS. As a result, the DGCS was able to monitor the virtual environment very
similarly, based on the latitude/longitude data of the UAV flying in Gazebo. Comparing
the coordinate distance between the UAV moving in Gazebo and the UAV monitored on
DGCS, DGCS was able to monitor the UAV with an average error rate of about 6.75cm.
Moreover, we measured the data transmission performance to evaluate the efficiency of
the DGCS. The data used for the measurement is flight status data, including battery,
status, and GPS, and is 1KB in size. Fig. 5 shows the results of measuring the data
transmission performance of the DGCS in overview architecture (Fig. 1).
The measurement results showed that DGCS took about 18ms to monitor the UAV’s
status. This is a performance that can monitor the status of the UAV every 9cm when the
UAV moves at about 5m/s. Also, it took about 25ms to remotely control the UAV from
DGCS. This is a performance that allows for the transmission of about 40 commands
per second. As a result, we confirmed that DGCS has the efficiency to enable monitoring
and remote control with less delay time.
(a) Latitude Comparison (b) Longitude Comparison
Figure 4: The Comparison of the UAV’s Latitude/Longitude in Gazebo and the Monitoring Results from
DGCS (Average Error Rate about 6.75±1cm )
Figure 5: The results of DGCS data transmission performance in the UAV environment
5. Conclusion
We proposed a DGCS for monitoring the status of UAVs status with less delay time.
The DGCS utilized digital twin for precise monitoring and to reflect rapidly changing
environments. The DGCS was able to monitor the simulation UAVs with an average
error rate of about 6.75 cm. Also, The DGCS could monitor UAVs at a rate of about
18ms for 1KB of data and control UAVs at a rate of about 25ms. Through this, we
confirmed that DGCS is effective for real-time monitoring of UAVs. In future work, we
plan to add automated remote control to DGCS. Also, we plan to add simulation and
prediction features using the digital twin.
Acknowledgments
This results was supported by "Regional Innovation Strategy (RIS)" through the National
Research Foundation of Korea(NRF) funded by the Ministry of Education(MOE)(2021RIS-
003)
References
[1] W. Liu, T. Zhang, S. Huang, K. Li, A hybrid optimization framework for uav
reconnaissance mission planning, Computers & Industrial Engineering 173 (2022)
108653.
[2] W. Sun, L. Dai, X. Zhang, P. Chang, X. He, Rsod: Real-time small object detection
algorithm in uav-based traffic monitoring, Applied Intelligence (2021) 1–16.
[3] L. Wang, Z. Zhang, Automatic detection of wind turbine blade surface cracks
based on uav-taken images, IEEE Transactions on Industrial Electronics 64 (2017)
7293–7303.
[4] J. Scott, C. Scott, Drone delivery models for healthcare (2017).
[5] Qgroundcontrol, Accessed: October 2023. URL: https://qgroundcontrol.com.
[6] Mission planner, Accessed: October 2023. URL: https://ardupilot.org/planner/.
[7] Apm planner, Accessed: October 2023. URL: https://ardupilot.org/planner2/.
[8] X. Liang, S. Zhao, G. Chen, G. Tong, L. Jiang, W. Zhang, Design and development
of ground control system for tethered uav, in: 2019 IEEE International Conference
on Unmanned Systems (ICUS), 2019, pp. 291–296. doi:10.1109/ICUS48101.2019.
8996006 .
[9] S. R. Haque, R. Kormokar, A. U. Zaman, Drone ground control station with
enhanced safety features, in: 2017 2nd International Conference for Convergence in
Technology (I2CT), 2017, pp. 1207–1210. doi:10.1109/I2CT.2017.8226318 .
[10] J. C. del Arco, D. Alejo, B. C. Arrue, J. Cobano, G. Heredia, A. Ollero, Multi-uav
ground control station for gliding aircraft, in: 2015 23rd Mediterranean Conference on
Control and Automation (MED), 2015, pp. 36–43. doi:10.1109/MED.2015.7158726 .
[11] M. Grieves, Digital twin: manufacturing excellence through virtual factory replica-
tion, White paper 1 (2014) 1–7.
[12] F. Tao, M. Zhang, Digital twin shop-floor: a new shop-floor paradigm towards smart
manufacturing, Ieee Access 5 (2017) 20418–20427.
[13] S. Haag, R. Anderl, Digital twin–proof of concept, Manufacturing letters 15 (2018)
64–66.
[14] E. J. Tuegel, A. R. Ingraffea, T. G. Eason, S. M. Spottswood, Reengineering aircraft
structural life prediction using a digital twin, International Journal of Aerospace
Engineering 2011 (2011).
[15] Mapbox, Accessed: October 2023. URL: https://www.mapbox.com/.
[16] Openweather, Accessed: October 2023. URL: https://openweathermap.org/.
[17] N. Koenig, A. Howard, Design and use paradigms for gazebo, an open-source multi-
robot simulator, in: 2004 IEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS) (IEEE Cat. No.04CH37566), volume 3, 2004, pp. 2149–2154
vol.3. doi:10.1109/IROS.2004.1389727 .