Quadcopters are susceptible to internal and external in uences, many of which may lead to faults. To ensure a safe and reliable ight, the quadcopter needs to recover autonomously from faults. However, existing approaches mainly rely on parametrical faults or require a prede nition of possible faults which is not realistic for a complex realworld scenario. The recovery from unforeseen faults and structural faults like a failing engine is still an open research gap. Hence, in this paper, a concept for the automated recon guration, i.e. the automated recovery from a fault, which only uses information about non-faulty system behavior and is able to handle structural changes is presented. From the information about non-faulty behavior a non-faulty system model is created using established machine learning methods. Thus, faults are detected by learned model and no pre-de nition of faults is needed. The system structure is modeled using a logical calculus which allows for modeling available system parts and the causal coherences between these. The approach is applied to a simulation of a quadcopter which underlies a structural fault. It is shown that the approach extends the capabilities of a quadcopter to handle faults autonomously and ensure stability and reliability.
Unmanned Aerial Vehicles (UAVs) are an emerging technology of great interest in military and civil applications [21,11]. The market for UAVs has emerged especially in the last years and is expected to rise continuously [11,7]. One type of UAVs, the quadcopters which consist of four rotors that can be controlled independently from each other, is in the scope of most research studies [21]. Since quadcopters operate in an open world, the requirements towards reliability and safety are very high. Today, due to many safety concerns, the usage of quadcopters is subject to massive restrictions [11]. It stems from a fact that minor UAV fault can lead to major consequences, including huge damage to humans or environment. Hence, quadcopters need to be designed to be robust to environmental disturbances and tolerant towards internal faults. Currently, controllers used for Cyber-Physical Systems, including quadcopters, are static and at most applicable to a prede ned set of faults [24]. An automated recon guration of the control is needed to maintain a stable ight in presence of faults [5]. Recon guration is the task of recovering a valid system state after a fault has occurred [3]. However, the automated recon guration for quadcopters is still an open research gap due to some unanswered research questions, two of which follow.
RQ1: How can automated recon guration handle unforeseen faults? Existing approaches on fault-tolerant control are mostly based on an enumeration of known faults and the storage of control instructions speci c to these faults [21]. However, when it comes to unknown faults, these approaches can no longer guarantee stable control. Hence, in this paper, a concept which needs no information about known faults but only works on information about non-faulty behavior is presented. Quadcopters contain a multitude of sensors, continuously logging data during the ight. This huge amount of data can be analyzed in an intelligent way using Machine Learning (ML) methods. These methods enable learning a model of the system from historical data and can then be used to detect anomalous behavior, which might indicate the presence of faults [18]. The data used for training contains only measurements from non-faulty ights, thus deviations from non-faulty behavior are handled as candidates for faults.
RQ2: Which formalism is able to handle structural faults? Control theory in general is concerned with a static system model. However, when major faults like an engine failure or a rotor ripping o occur, the system is no longer representative, leading to the control becoming invalid. Logical reasoning can be used here to draw conclusions about the impact of a fault as well as still available and functional components and actions [3]. Thus, new control instructions can be determined even in the presence of major faults.
The contribution of this paper is twofold: (
Please note that the scope of this paper is not to handle one given fault in an optimal way but to present a concept that restores a safe ight in the presence of unforeseen faults and disturbances. The paper is structured as follows: First, in section 2 the related work is presented and discussed. Then, the solution concept is presented in section 3. In section 4, using a simulation of a quadcopter, the applicability of the solution concept is evaluated. 2
Most research on quadcopter control is concerned with fault-tolerant control
(FTC) [22,17]. The goal of FTC is to maintain a stable ight in the presence of
wind as well as actuator and sensor faults. Therefore, the quadcopter is described
in quantitative terms [26] using the equations
x(t + 1) = f (x(t); u(t))
y(t) = g(x(t); u(t))
(
Lunze [14] presented a concept towards recon gurable control for UAVs using overdetermined sets of equations. However, this approach requires explicit modeling of the quadcopter behavior and thus a large amount of expert knowledge. Wang et al. [27] presented a combination of classical control and constraint satisfaction. The scope of this work is slightly di erent: no faults are handled but an optimal control for a given path is searched. Chen et al. [6] presented an approach to recon guration of actuator faults by an advanced estimation procedure. However, the faults need to be modeled explicitly. Thus, no unknown faults can be handled. Adaptive control algorithms, as presented by Huynh et al. [10] among others, are concerned with continuous disturbances like wind or varying parameters. Unknown or structural faults cannot be handled [21]. Robust control algorithms as presented by Thanh et al. [23] and Ton et al. [25] handle parametric uncertainties and are even adaptable to nonlinear disturbances. However, no structural faults can be handled [21].
The recon guration concept presented here can be seen as an extension to classical control theory. The goal is not to determine control instructions for an optimal ight, but to identify the necessary actions to recover a stable ight in the presence of faults.
The goal of automatic recon guration is to manipulate the system inputs to restore valid system behavior [3]. To perform automatic recon guration, some kind of redundancy is necessary [5]. This can be either physical redundancy, e.g. duplicate components or sensors, or analytical redundancy, i.e. information about the relation between di erent values measured by the system. Quadcopters in general contain numerous redundancies to ensure safety and reliability requirements are ful lled. Typical examples for physical redundancy are multiple engines, multiple batteries or multiple sensors. Analytical redundancy can be asserted through knowledge about coherences and relations between sensor and actuator values.
Data Control 1. Recovering stable flight
Anomaly No Anomaly 2. Reconfiguration of Control Unit
Faults can be di erentiated using the component they concern. Thus, we divide actuator faults which are usually modeled as loss of e ectiveness of one rotor, sensor faults which are modeled as sensors returning wrong values and strutuctual/system faults which a ect system components like engines or batteries. Actuator faults can be handled using robust control methods, sensor faults usually are handled using Kalman lters [15]. In general, not every possible fault and its consequences is known a-priori because quadcopters operate in a non-deterministic environment, many di erent factors like wind speed, rain and air humidity have an impact on the behavior of the quadcopter. Thus, foreseeing every possible consequence of environmental in uences on the quadcopters behavior and enumerate every possible fault is impossible. The goal of recon guration is to adapt the rotor speeds to recover a stable ight in case of every unforeseen fault or at least to perform a safe emergency landing.
The basic concept of automatic recon guration for quadcopters is shown in Figure 1. The recon guration system operates while the quadcopter is ying and continuously checks for deviations. First, the data delivered by the sensors of the quadcopter is compared to the learned models, checking for deviations and the presence of symptoms. If no symptoms are present, no further actions are needed and neither the control unit nor the system goal are changed. In case of at least one symptom, the recon guration unit rst estimates the extent of the deviation. Then, recon guration is performed in two steps [5]: 1. A set of actions that moves the quadcopter back into a valid ight is searched and applied directly. 2. A controller that stabilizes the quadcopter in its valid ight in the presence of faults (e.g. a di erent control structure due to a failure of one sensor) is determined using well known controller design methods.
In some cases, the recon guration to a valid ight is no longer possible since the damage is too high. Then, during the rst recon guration step, actions to perform an emergency landing or a return to launch, if possible, are applied until the quadcopter has landed. The control unit is not recon gured, since a stable ight cannot be recovered. 3.1
To enable the recon guration unit to handle structural faults, it needs to reason about the consequences of a fault. Therefore, information about the causal coherences, i.e. the impact of a change in one component to other components is needed. Logic is used widely in Arti cal Intelligence since it allows for modeling causal coherences and drawing logical conclusions about the system [20]. Basic physical and mathematical knowledge can be modeled in logic to support the recon guration unit in its decision making [13]. Using the logical calculcus Satis ability Modulo the Theory of Linear Arithmetic [4], also continuous variables (e.g. rotor velocity, wind speed, ...) can be modeled.
1 01 02
0 3
To model the causal coherences of the quadcopter, rst, the system topology is analyzed. Thus, the coherences between the components can be described in terms of a logical calculus. Therefore, the fuselage 0 and each rotor r1; r2; r3; r4 are modeled as nodes, the set of all nodes is represented by N . Since every rotor is connected to the fuselage, the edges are represented by E = fe01; e02; e03; e04g. Fig. 2 shows the resulting graph for a quadcopter. The behavior of every component, i.e. the rotors and the fuselage, is monitored using component models [8]. Today, due to sophisticated learning methods, these models no longer need to be created manually but can be learned [19,2]. Comparing the current behavior of a component to the model, deviations indicating faults are detected. This information is encoded by a binary assignment ! : N ! f>; ?g which is true if the component behaves non-faulty and false otherwise. A connection between two components may only be used if the components are non-faulty, i.e. be+ij ) !(i) ^ !(j) 8eij 2 E:
Thus, component faults are directly taken into account by the recon guration unit. Structural changes due to a component fault (e.g. a failing engine) can be represented by the component model of the corresponding rotor showing a deviation to the current behavior. Hence, the recon guration unit identi es the necessary changes under consideration of the impact of the fault.
Therefore, every components behavior is monitored using sophisticated machine learning methods utilizing a normal behavior system model. Here, a modeling formalism which transforms the data into a black- or gray-box representation is used. Such a structure does not explicitly model each observation but creates a new representation based on the normal behavior data. Depending on the formalism used, various measures describe how well the current status ts into the model. An example well-known algorithm is Self-Organizing Map [12], a type of neural network. The measure of tting in this case is the quantization error, which is the di erence between the current status location mapping and the bestmatching neuron neighborhood in the model. If this quantization error is high, the component is assumed to behave anomalously, so a symptom is reported to the recon guration unit to trigger further actions.
For the creation of these models only data about non-faulty system behavior is required. Thus, no fault modes or an enumeration of known faults has to be given. 3.3
As mentioned above, the recon guration is performed in two steps: One step to regain a stable ight and one step to maintain this stable ight. Whilst the second step can be done using well{known controller design methods, the rst step is still an open research gap [5].
For each rotor, the impact of acceleration and deceleration on the attitude and altitude of the quadcopter is modeled in terms of logical constraints, for example
If rotor r1 is accelerated, the pitch angle decreases. or If all rotors are accelerated proportionally, the height increases.
Thus, the impact of changing rotor velocities on the behavior of the quadcopter can be modeled. Based on this, the recon guration unit is able to choose an intelligent combination of rotor accelerations and decelerations to recover a stable ight, if possible. Otherwise, the recon guration unit tries to bring the quadcopter to a safe state (e.g. by a return to launch or an emergency landing).
To enable the recon guration method to change the velocity of the rotors, every connection is assigned with two binary variables that lead to an increase or decrease of the current velocity of the corresponding rotor. Therefore, for each edge e 2 E two binary variables be+ij ; beij that trigger an increase or decrease of the corresponding rotor are introduced. Thus, be+ij ! inc(rj ); beij ! dec(rj ). The predicates inc; dec indicate that the velocity of the corresponding rotor needs to be increased or decreased. How this is realized in detail needs to be de ned by an expert, e.g. that an increase is always realized by increase the velocity given a xed di erence or a percentage amount. To avoid that both variables are set to true at the same time (which would require a simultaneous increase and decrease of the velocity of one rotor) the constraint be+ij beij 8eij 2 E is needed.
Thus, the recon guration problem is modeled as a rst-order logic formula. If at least one symptom occurs, i.e. one component behaves anomalously, the recon guration unit determines for every connection, if the velocity of each rotor has to be increased, decreased, or does not have to be changed by setting the corresponding binary variable to true. Thus, the changes for recovering a stable ight are identi ed by a combination of acceleration and deceleration of rotor velocities. 4
This section presents the results of symptom generation and recon guration experiments. The symptom generation approach has been tested on real quadcopter data, to validate the approach as feasible in real-life scenarios. Recon guration experiments utilize a simulation to verify the outcome of recon gured and nonrecon gured fault scenario. The used simulation of the quadcopter is described in the appendix. The free variables in the logical formula created as described above are assigned with the current values of the sensors. Then, the formula is checked for satis ability to determine which rotors needs to be accelerated and which need to be decelerated to recover a stable ight in the presence of faults using the Z3 solver [16]. 4.1
To evaluate the concept, we show that it is indeed possible to detect anomalies in quadcopter behavior using machine learning-based modeling formalisms. Self-Organizing Map (SOM) model formalism is used to perform a preliminary analysis and investigate whether the methods are feasible to detect anomalies in quadcopter ight. We consider an approach feasible for symptom generation if it is possible, at least partially, to di erentiate between normal behavior and anomalous behavior using the model.
The symptom generation is performed on data from an industrial drone, to ensure the rst step of the concept is viable in real-life applications. Quadcopters are operated using the PX4, an open source ight control software for quadcopters and other unmanned vehicles. It allows logging device inputs (sensors etc.), internal states (CPU load, attitude etc.) and log messages. The log structure requires that sensors are organized within prede ned sensor groups, however the sampling rates, and therefore timestamps, of di erent sensor groups are independent from each other. It is therefore only possible to match the values from one sensor group at a time. The SOM model of normal behavior has been learned from chosen sensor logs of quadcopter ights where no faults occurred and the drone was considered to behave entirely correct. This model has then been used to detect anomalies in faulty ight logs.
It is expected that the anomalous behavior data at least partially overlaps the normal behavior in terms of quantization error, however, the results have shown that some sensor combinations generate a quantization error much higher in the case of faulty behavior than normal behavior. This outcome creates a perfect opportunity for symptom generation, where the maximum quantization error of normal behavior is used as the error threshold for a symptom, and a symptom is reported as soon as the error crosses the threshold. Figure 3 illustrates the di erence in quantization error over the initial ight period of ight: the error generated by SOM from the anomalous behavior data is decidedly higher than for normal behavior data. 4.2
The Engine Failure fault scenario focuses on a quadcopter ight, where one of the four engines that provide acceleration to the rotors fails mid- ight. The ight begins with a stable ight at the height of 10 meteres, until one engine fails and its corresponding rotors velocity decreases to 0. Only the three remaining rotors can m n i t h g i e h without recon guration
with recon guration 0 2 6
8 4 time in s be used to control the ight and the goal is to adjust their corresponding motor parameters in such a way that the drone does not su er a catastrophic failure. Classical control is not capable of handling the new situation with one less rotor: the control unit only recognizes a deviation in the attitude of the quadcopter and tries to adapt the velocity of the quadcopter rotor such that stability is regained. It is not taken into consideration, that one rotor is non-functional. Even in the presence of a fault, all of the engines are controlled similarly. No distinction between available and crashed engine can be made. The control unit uses all four rotors to adapt to the deviation, even though only three rotors are still available. Thus, no stable ight can be regained leading to a crash of the quadcopter. The recon guration method based on logical calculus enables the quadcopter to handle the fault and avoid a crash. First, the faulty component, in this case the failing engine is identi ed using the component models. Then, the recon guration unit calculates new control instructions to stabilize the ight by increasing the velocities of the rotors which are still available. However, a stable ight cannot be regained since the disturbance of a failing engine is to severe. Thus, an emergency landing is performed.
The simulated behavior of quadcopters equipped with classical control approach and recon guration approach is shown in Figure 4. Using the classical control, which does not adapt to the changed structure of the quadcopter, the quadcopter crashes within approximately 2 seconds. Since the quadcopter accelerates while falling, the velocity when touching the ground is around 13:4 meters per second which can lead to massive damages of the quadcopter and the surroundings. When the recon guration is enabled, the fall of the quadcopter is decelerated. The quadcopter touches the ground after approximately 8:4 seconds with a velocity of approximately 1:3 meters per second. Thus, damage to the quadcopter and the surrounding can be reduced signi cantly. 5
Quadcopter ights are susceptible to internal disturbances, like failing components, as well as environmental disturbances like winds. Today, FTC is commonly used to enable a quadcopter to maintain a stable ight even in the presence of faults. However, FTC is focused on handling numerical faults, which, in addition, often have to be prede ned. Thus, major faults which cause structural changes of the quadcopter cannot be handled by classical FTC. Therefore, this paper presents a concept for the automated recon guration to enable quadcopters to handle structural and unforeseen faults. The concept is based on the combination of a logic-based recon guration method. Faults are detected as deviations from models which are learned from non-anomalous behavior. Based on this information, recon guration is initiated { if necessary. During recon guration, stable ight of a quadcopter is described in terms of a logical calculus which allows for modeling condition of a stable ight and requirements to recover a stable ight, if possible. The approach is evaluated using an interactive simulation of a quadcopter. One of the four engines failing represents the structural fault. It is shown that without recon guration, the quadcopter crashes within 2 seconds and the velocity when touching the ground is high. With recon guration, the quadcopter touches the ground after 8 seconds with a far lower velocity.
Future work will focus on further fault scenarios like failures leading to a reduced engine performance or failing battery cells to prove the applicability of the concept. Then, also the scalability of the approach will be examined by using more detailed models.
This work was founded by the Fraunhofer Cluster of Excellence "Cognitive Internet Technologies".
A
Evaluation is performed using a simulation implemented in Modelica [9] is used. The ight of the quadcopter is described as a state space model (referring to [1]) = _ _ a1 + _a2 r + b1U2 = _ _ a3
_a4 r + b2U3 = _ _a5 + b3U4 x = (cos( ) sin( ) cos( ) + sin( ) sin( ))U1=m y = (cos( ) sin( ) sin( )
sin( ) cos( ))U1=m z =
g + (cos( ) cos( ))U1=m with
Ixx ; a4 = 42); U3 = d( 12 +
32); 1 + Jr
2
; a5 =
3 +
Ixx
4;
Iyy ;
(
Jr 8:5 10 4 kg m2 rotor inertia Ixx = Iyy 1:96 10 2 kg m2 quadcopter inertia around x/y-axis
Izz 2:62 10 2 kg m2 quadcopter inertia around z-axis l 0:21 m length of arms b 9:29 10 5 N s2 thrust coe cient d 1:1 10 6 N m s2 drag coe cient
Table 1. Values for the parameters of the quadcopter
The values of the parameters are shown in Table 1 (also referring to [1]). The
inputs of the system are represented by the velocities of the rotors 1; 2; 3; 4.
10. Huynh, M.Q., Zhao, W., Xie, L.: L 1 adaptive control for quadcopter: Design and
implementation. In: 2014 13th International Conference on Control Automation
Robotics & Vision (ICARCV). pp. 1496{1501. IEEE (2014)
11. Joshi, D.: Drone technology uses and applications for commercial, industrial
and military drones in 2020 and the future. Businees Insider (2019), https://
www.businessinsider.com/drone-technology-uses-applications?r=DE&IR=T,
called 22.04.2020
12. Kohonen, T.: Self-Organizing Maps, Springer Series in Information Sciences,
vol. 30. Springer-Verlag, Berlin Heidelberg, 3 edn. (2001)
13. Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines
that learn and think like people. Behavioral and brain sciences 40 (2017)
14. Lunze, J.: From fault diagnosis to recon gurable control: A uni ed concept. In:
2016 3rd Conference on Control and Fault-Tolerant Systems (SysTol). pp. 413{
421. IEEE (2016)
15. Moghadam, M., Caliskan, F.: Actuator and sensor fault detection and diagnosis
of quadrotor based on two-stage kalman lter. In: 2015 5th Australian Control
Conference (AUCC). pp. 182{187. IEEE (2015)
16. de Moura, L., Bj rner, N.: Z3: An e cient SMT solver. In: International
Conference on Tools and Algorithms for the Construction and Analysis of Systems. pp.
337{340. Springer (2008)
17. Nguyen, N.P., Hong, S.K.: Fault diagnosis and fault-tolerant control scheme for
quadcopter uavs with a total loss of actuator. Energies 12(
Computers & chemical engineering 27(