An Unmanned Aerial Vehicle (UAV) is an aircraft that does not carry humans on board. UAVs are recognized as key components in various fields such as reconnaissance [1], detection [2,3], and delivery [4]. Since UAVs fly without humans on board, they are operated through remote control and autonomous flight. Ground Control System (GCS) is necessary to ensure the flight management and safety of UAVs flying at a distance from the pilot.
A GCS enables the pilot to monitor the flight path, status, and sensor data of UAVs from the ground. Through the GCS, the pilot can understand the current status of UAVs and, in case of danger, can remotely control UAVs. For precise status monitoring and remote control in emergency situations, GCS can be more efficient if it is capable of modeling environments identical to those of UAVs in flight.
Commercial GCS [5,6,7] for existing UAVs typically represent status and sensor data only as parameters. Previous studies [8,9,10] on GCS proposed systems for controlling and monitoring single or multiple UAVs. However, these studies have difficulty monitoring the status of UAVs considering rapidly changing weather and surrounding environments, making it difficult to guarantee the safety of the UAV. This paper proposes a Digital twin-based Ground Control System (DGCS) for monitoring the status of UAVs. The DGCS can monitor the status of UAVs in a 3D-modeled virtual environment by integrating a digital twin with the GCS. A 3D-modeled virtual environment can provide users with the current status of a UAV in flight, reflecting information about rapidly changing weather and surrounding environments. Through this, the DGCS can monitor the status of UAVs more precisely with less delay time, ensuring its safety.
To evaluate the DGCS, we measured its accuracy and efficiency. To measure accuracy, we measured how precisely the DGCS could display the flight status of a UAV operated in a simulation environment. As a result, the DGCS was able to monitor UAVs with a less delay time with an average error rate of about 6.75cm. To measure efficiency, we measured the data transmission performance in the environment where the DGCS operates. As a result, the DGCS could monitor about 1MB of status data at a rate of about 18ms, and it could transmit 1MB of command data to the UAV at a rate of about 25ms.
The contributions of this study are as follows: (1) It is a GCS capable of precise monitoring. The proposed system, utilizing digital twin technology, enables precise monitoring in a 3D-modeled environment within virtual space, considering terrain, weather conditions, etc. (2) It is a scalable GCS. By using a digital twin, the proposed system can add various functions such as simulation and prediction within virtual space.
Since UAVs fly over a wide range and perform missions, a remote control and monitoring system is essential. GCS is a system that allows controlling in-flight UAVs from the ground.
GCS consists of hardware and software. The hardware consists of electronic devices that establish data communication between the GCS and UAVs. The data communicated includes flight data from UAVs and control commands from the pilot. The software consists of the user interface. This user interface provides the current status of UAVs and parameters for control.
GCS can interface with single and multiple UAVs. UAVs transmit their current flight status and sensor data to the GCS. GCS can perform tracking and monitoring tasks based on the data received from UAVs. Also, the GCS provides a feature that allows pre-setting the flight path of UAVs. GCS allows pilots on the ground to remotely monitor and control UAVs, enhancing safety.
Digital twin was first proposed by Professor Grieves [11] from Michigan University. Grieves explained that a digital twin consists of a physical space, a virtual space, and the connection between the two spaces. The major feature of the digital twin is to model the physical space identically in the virtual space to enhance interaction [12].
Digital twin allows for the real-time monitoring of actual objects existing in the physical space within a virtual space. Also, in the virtual space, digital twin can simulate various situations for a particular object, predicting malfunctions and performance degradation [13,14]. Objects in physical space can be controlled based on predicted results.
Using digital twin allows for more precise monitoring and prediction than existing GCS through a modeled virtual space. Furthermore, automated remote control can be executed based on real-time UAV and surrounding environment data.
Currently, representative GCS include QGC (QGroundControl) [5], Mission Planner [6], and APM Planner [7]. These systems can interface with drones to monitor parameters, such as the drone's status data (Yaw, Pitch, Roll, etc.), and to create flight paths. However, in these systems, it is difficult to display sensor data additionally mounted on the UAV to the user depending on the situation, in addition to the remote measurement data. Also, since these systems are not monitored in a 3D virtual environment, precise monitoring is difficult.
Research is being conducted on the GCS for UAVs outside of the introduced commercial systems. Liang et al. [8] presented a GCS for tethered UAVs. This system displays the UAV status and mission information to the user through a GUI (Graphical User Interface). Haque et al. [9] presented a GCS, which includes a micro-controller to allow users to easily control the UAV. This system integrates software to display no-fly zones and weather parameters to the user. Arco et al. [10] presented a GCS for monitoring the missions of multiple UAVs. This system is capable of performing HIL (Hardware in the Loop) and allows the real-time monitoring of the status of multiple UAVs through a GUI.
Table 1 shows the analysis of related work. Previous works have presented GCS for controlling and monitoring single and multiple UAVs. However, in previous works, it is difficult to monitor and control the status of the UAV in a rapidly changing flight environment because they are difficult to reflect terrain and weather data. Therefore, this paper presents a DGCS that can monitor and control UAVs in a digital-based virtual environment by reflecting terrain and weather data.
This paper proposes a Digital twin-based Ground Control System (DGCS) for monitoring of UAVs. The DGCS uses digital twin for ground control. Through the digital twin, the DGCS enhances the interaction between the physical space (actual UAV) and the virtual space (control system). Fig. 1 shows the overall architecture of the DGCS. As shown in Fig. 1, the DGCS is composed of a physical space, a digital twin (communication between the two spaces), and a virtual space. The physical space consists of the UAV, the automatic controller (Pixhawk), and the onboard computer for communication. The digital twin is composed of a database to manage the UAV of the physical space and supports communication between the physical and virtual spaces. The virtual space consists of 3D-modeled terrain, simulation modules, and a User Interface (UI) that can interact with the user. The UI represents DGCS, which allows users to monitor and remotely control the UAV.
The operation sequence of DGCS is as follows:
1. The onboard computer collects the status and sensor data of the UAV.
2. The onboard computer transmits the collected data to the digital twin.
3. The digital twin manages the status of single and multiple UAVs based on ID. 4. The virtual space represents the UAV in the modeled virtual environment based on the data stored in the digital twin. 5. Users can monitor the UAV status.
The environment to implement the DGCS used Windows 10, CPU Intel i7-11700, and RAM 32GB. In Fig. 1, the onboard computer in the physical space used a Raspberry Pi 4 model. The digital twin is implemented in Python, and communication between the two spaces uses WebSockets. The virtual space is implemented as a JavaScript-based web application, and Mapbox was used to create the 3D model. Fig. 2 shows the DGCS, the implemented GCS. Fig. 2 x shows the 3D-based terrain and features implemented using Mapbox [15]. Fig. 2 y shows the GPS (Global Positioning System) coordinates (longitude/latitude) and altitude of the UAV currently registered in the digital twin. In y, if the current UAV is not at risk of collision, it is marked in white; if it is passing over a building, it is marked in orange; and if it is flying at an altitude lower than the building, it is marked in red. Also, DGCS can represent weather data using the OpenWeather API [16].
The experimental environment for evaluating the DGCS is identical to the implementation in Section 3.2. To evaluate the accuracy of the DGCS, we experiment with whether the DGCS can monitor the information of the operating UAV in real time. For this experiment, we used Gazebo [17], which can simulate UAVs. Fig. 3 shows the results of an experiment whether the DGCS can monitor a UAV moving at about 5m/s in Gazebo. Fig. 3 (a) shows the path of the UAV in Gazebo. Fig. 3 shows the results of the DGCS monitoring the UAV in flight within Gazebo. As a result, we was confirmed that the DGCS can monitor the flight path of the UAV in flight within Gazebo.
Fig. 4 shows the results of comparing how accurately the DGCS monitored the GPS data from Fig. 3, categorized by latitude and longitude. Fig. 4a and 4b show a comparison of latitude and longitude data from UAVs flying within Gazebo and UAVs monitored by DGCS. As a result, the DGCS was able to monitor the virtual environment very similarly, based on the latitude/longitude data of the UAV flying in Gazebo. Comparing the coordinate distance between the UAV moving in Gazebo and the UAV monitored on DGCS, DGCS was able to monitor the UAV with an average error rate of about 6.75cm.
Moreover, we measured the data transmission performance to evaluate the efficiency of the DGCS. The data used for the measurement is flight status data, including battery, status, and GPS, and is 1KB in size. Fig. 5 shows the results of measuring the data transmission performance of the DGCS in overview architecture (Fig. 1).
The measurement results showed that DGCS took about 18ms to monitor the UAV's status. This is a performance that can monitor the status of the UAV every 9cm when the UAV moves at about 5m/s. Also, it took about 25ms to remotely control the UAV from DGCS. This is a performance that allows for the transmission of about 40 commands per second. As a result, we confirmed that DGCS has the efficiency to enable monitoring and remote control with less delay time.
We proposed a DGCS for monitoring the status of UAVs status with less delay time.
The DGCS utilized digital twin for precise monitoring and to reflect rapidly changing environments. The DGCS was able to monitor the simulation UAVs with an average error rate of about 6.75 cm. Also, The DGCS could monitor UAVs at a rate of about 18ms for 1KB of data and control UAVs at a rate of about 25ms. Through this, we confirmed that DGCS is effective for real-time monitoring of UAVs. In future work, we plan to add automated remote control to DGCS. Also, we plan to add simulation and prediction features using the digital twin.
This results was supported by "Regional Innovation Strategy (RIS)" through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(MOE)(2021RIS-003)