=Paper=
{{Paper
|id=Vol-2783/paper09
|storemode=property
|title=Online Parameter Estimation and Optimal Input Design
|pdfUrl=https://ceur-ws.org/Vol-2783/paper09.pdf
|volume=Vol-2783
|authors=Barzin Hosseini,Johannes Diepolder,Florian Holzapfel
|dblpUrl=https://dblp.org/rec/conf/birthday/HosseiniDH20
}}
==Online Parameter Estimation and Optimal Input Design==
https://ceur-ws.org/Vol-2783/paper09.pdf
Online Parameter Estimation and
Optimal Input Design ? ??
Barzin Hosseini[0000−0002−5767−0115] , Johannes Diepolder[0000−0002−5962−6126] ,
and Florian Holzapfel[0000−0003−4733−9186]
Institute of Flight System Dynamics, Technische Universität München,
Garching bei München, Germany
{barzin.hosseini,johannes.diepolder,florian.holzapfel}@tum.de
Abstract. In flight system identification, flight test data is used to es-
timate model parameters. The aircraft is excited during the flight with a
set of inputs (maneuvers) to generate the data for parameter estimation.
Different approaches exist to design the inputs which result in an ade-
quate information content in the acquired data during the flight tests. In
most cases, inputs are applied to the aircraft which are designed based
on a priori knowledge about the system. In this paper we investigate a
method to update the model parameters during the flight and update
the inputs onboard the aircraft. This method can increase the achieved
parameter accuracy and is expected to reduce the required time for the
flight test program, and therefore, also the program cost.
Keywords: Parameter estimation · Optimal input design · Optimal con-
trol · System identification.
1 Introduction
In flight vehicle system identification, a mathematical model is deducted from
recorded flight data using a series of test maneuvers. If a fixed model structure is
assumed, the system identification task narrows down to a parameter estimation
problem. It has been shown that the type of inputs applied to an aircraft has a
significant influence on the accuracy of the parameter estimates [11, 12]. In this
context, it is desirable to apply inputs to the dynamic system which are expected
to maximize the information content in the flight test data. As such, optimal
inputs can help to reduce the flight test time while resulting in more accurate
parameter estimates. This reduces the overall cost of a system identification
project, specially when flight tests are expensive to perform and flight time
is limited on the flight vehicle under investigation (e.g. cruise missiles). This
problem can be formulated as an optimal control problem with a special cost
?
This work is supported by the German Federal Ministry for Economic Affairs and
Energy as part of the LuFo program (grant-ID 20Q1719D).
??
Copyright c 2020 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
Online parameter estimation and optimal input design 129
function. However, an initial model (with initial values for the parameters) is
required to pose and solve the optimal control problem. Since determining the
parameter values in the model is the primary goal of the flight tests and the
subsequent parameter estimation, accurate parameter values often do not exist
when designing the experiments. Different approaches have been suggested to
solve this problem, such as a robust approach to design the optimal inputs, in
which the uncertainty of the optimal inputs can be considered in the input design
process [5]. In this paper, we investigate the application of online parameter
estimation to adapt the optimal inputs during the runtime of the experiment.
Besides flight testing for system identification and parameter estimation, in-
direct adaptive control is also an important use case for onboard adaption of
optimal input design for parameter estimation. In such cases, the controller is
adjusted based on the change in the system parameters. Therefore, maneuvers
are pursued which lead to more accurate parameter estimates and therefore a
better performing controller. Many previous studies such as [8] have investigated
the topic of onboard optimal input design in connection with indirect adaptive
control.
The problem discussed in this paper can be formulated as a Dual-Mode
Batch-to-Batch Optimization as presented in [4]. Based on this approach, it is
possible to assess our solution method and quantify the possible loss of optimal-
ity. This is however beyond the scope of the present study and will be investigated
in future work.
2 Problem Definition
The problem in parameter estimation is to find the parameter values in a fixed
model structure
θ̂ = θ̃(Z, u) . (1)
In the above equation, θ̂ is the vector of the parameter estimates, θ̃ is the
estimator and Z and u are the measured outputs and inputs. In this paper, we
limit our study to linear systems of the following form
ẋ = Ax + Bu, (2)
y = Cx + Du, (3)
z(ti ) = y(ti ) + v(ti ), i = 1, 2, . . . , N, (4)
where x is the vector of the system states, y is the output vector, u is the input
vector and A, B, C and D are the system-, input-, output- and feed-through
matrices. Moreover, v is a zero-mean Gaussian white noise sequence with the
covariance matrix R.
It can be shown that the Fisher information matrix M provides an indica-
tion about the information content in the gathered flight test data during the
experiment
130 B. Hosseini et al.
N T
X ∂y(ti ) −1 ∂y(ti )
M= R . (5)
i=1
∂θ ∂θ
The inverse of the Fisher information matrix, the Dispersion matrix D =
M −1 is the theoretical lower bound for the covariance matrix of the parameter
estimates if an unbiased estimator is used [7, 12]. The square roots of the diag-
onal elements of the Dispersion matrix D are called Cramér-Rao bounds and
are the theoretical lower bounds for parameter estimate standard deviations.
If an asymptotically efficient estimator is used, the Cramér-Rao bounds are an
estimate for the parameter estimate standard deviations.
Since the Cramér-Rao bounds and the Dispersion matrix do not depend on
the measurements, an initial model and initial values for parameters can be used
to design inputs that minimize the remaining uncertainty in the parameters. In
this study, we choose to minimize the trace of the Dispersion matrix which is the
sum of the estimate for the parameter variances under consideration of practical
state and input constraints. The optimal control problem (OCP) can be written
as
tr M −1
minimize
u
subject to ẋ = f (u, x, θ) ,
x ∈ [xlb , xub ] ,
(6)
u ∈ [ulb , uub ] ,
θ = θ0 ,
t ∈ [0, T ],
where T is the duration of the maneuver.
Since the estimation of the parameter values is the ultimate goal of the flight
test campaign, no precise values of the parameter estimates are available before
the flight tests. A common approach is to perform flight tests with suboptimal
inputs or optimal inputs based on inaccurate parameters, record the data and
perform offline parameter estimation. The new parameter values are used to de-
sign new inputs which are used for subsequent flight tests. This becomes a costly
iterative process. Robust optimal control methods have been used in [5] to solve
this uncertain optimal control problem. However, as the aircraft is excited over
time, more information becomes available about the aircraft dynamics which
can then be used to estimate more accurate parameter values. The new param-
eter values can be used to adjust the maneuvers onboard the aircraft. To reduce
the required time between the maneuvers, we suggest estimating the parameters
online while new measurements become available.
3 Solution Method
The solution method applied in this study consists of an Extended Kalman Filter
(EKF) for the online parameter estimation. The approaches for designing the
optimal inputs are based on [6].
Online parameter estimation and optimal input design 131
3.1 Online Parameter Estimation
Different methods exist for online flight vehicle parameter estimation. These
methods allow to update the parameter values onboard the aircraft as new mea-
surements become available rather than recording the measurements over a time
period and analyzing them in batches. Online parameter estimation methods
can be formulated in time and frequency domain as described in [10, 12]. In this
study we use an EKF for estimating the states and parameters in realtime. The
implementation in this study is based on [12]. The state vector of the dynamic
system (2) is augmented with the system parameters as follows
x
xa = . (7)
θ
The dynamics of the original system states is governed by (2). Since we
assume the parameters to be constant, their dynamics is governed by
θ̇ = 0. (8)
The governing equations for the augmented system become
ẋa = Aa xa + B a u, (9)
y = C a xa + Du, (10)
z(ti ) = y(ti ) + v(ti ), i = 1, 2, . . . , N, (11)
(12)
with the initial conditions
E[xa (0)] = x̄a,0 , (13)
E{[xa (0) − x̄a,0 ][xa (0) − x̄a,0 ]} = P a,0 , (14)
and
A(θ) 0 B(θ) x̄
, C a = B(θ) 0 , x̄a,0 = 0 ,
Aa = , Ba = (15)
0 0 0 θ
where x̄a,0 is the mean value of the augmented states at the initial time point
and P a,0 is its respective covariance matrix.
The dynamic system described above is not linear anymore since the new aug-
mented states which include the parameters are elements of the system matrix A
and are multiplied with the state vector x. Therefore, the equations governing
the nonlinear system (9) can be written as
132 B. Hosseini et al.
ẋa = f (xa , u) , (16)
y = g(xa ) , (17)
z(ti ) = y(ti ) + v(ti ), i = 1, 2, . . . , N. (18)
The parameters of the system (2) are now states of the augmented sys-
tem (16) and can be estimated via an EKF as described in [12]. In addition to
the parameter values, an estimate for the parameter covariance matrix is also
computed by the Kalman filter.
3.2 Optimal Input Design
As mentioned above, optimal inputs for parameter estimation are determined by
solving the optimal control problem described in (6). The cost function of the
mentioned problem is non-convex and previous studies in [6] have shown that
an initial guess near the optimal point is required to achieve a solution via the
direct method for optimal control (using gradient based optimization). The two
stage optimization method of [6] is applied to solve the optimal control problem
stated in (6). First, a dynamic programming method with a relatively rough dis-
cretization is applied to generate an initial guess for the optimal control and the
respective system response. In a second step the direct method for optimal con-
trol (see [1]) is initialized with the results of the first step and applied to find the
final solution. In this case the original optimal control problem (6) is transcribed
into a nonlinear programming problem and solved with a gradient based solver.
We use an optimal control and parameter estimation toolbox, Falcon.m [13], de-
veloped at the Institute of Flight System Dynamics of TU Munich for the second
optimization stage. The details of the solution methodology and implementation
for designing optimal inputs based on an initial model can be found in [6].
3.3 Onboard Adjustment of the Inputs
As stated before, the optimal inputs computed offline are based on an initial
model with an initial set of parameters. As shown in Fig. 1 an optimal maneuver
is designed offline with the initial values of the parameters. This maneuver is
injected during the flight. An EKF updates the parameter values during the
optimization as new data becomes available.
After each maneuver, the gradient based optimization is repeated with the
updated parameter values to adapt the optimal inputs. Thereby, the optimal
input generated with the last set of parameters and the corresponding system
states are used to initialize the optimization with the direct method for optimal
control. Similarly, each EKF run for online parameter estimation is initialized
by previous estimates for the parameters, their covariance matrix and the state
parameter correlations. In the first parameter estimation run, we initialize the
covariance matrix of the parameters by a diagonal matrix with a large value
Online parameter estimation and optimal input design 133
Pre-Flight In-Flight
Dynamic Online Parameter Onboard
Initial Model Estimation
Programming Maneuver
Maneuver Adaption
Direct Method Injection
for Optimal Check
Control Not Met
Iteration
Met
End
Criteria
Fig. 1. Algorithmic steps of the pre-flight and in-flight maneuver optimization.
(here 105 ) on its diagonal. No correlation is assumed between the parameters
and the states of the system in the initial covariance matrix of the augmented
states (for the first maneuver). The initial state covariance matrix is considered
as a tuning parameter and should be chosen considering the accuracy of the
aircraft’s navigation system and controller.
4 Numerical Results
A short period model of an aircraft is used to illustrate the proposed method in
this study. The linear model was featured in multiple previous studies such as
[2, 6, 9, 11]. The model is described as
α̇ Z 1 α Z
= α + η η, (19)
q̇ M α Mq q Mη
where α is the angle of attack, q the pitch rate, and η the elevator deflection.
The output equation is
y1 (t) 1 0 α(t)
= . (20)
y2 (t) 0 1 q(t)
The measurement equations of the system are:
ym1 (i) 1 0 y1 (ti ) v1 (ti )
= + , i = 1, 2, . . . , N, (21)
ym2 (i) 0 1 y2 (ti ) v2 (ti )
v1 (ti )
v(i) = . (22)
v2 (ti )
In the above equation, v(ti ) is the i-th realization of a zero mean white
Gaussian random process with the following covariance matrix
2.0 0.0
R= . (23)
0.0 1.0
134 B. Hosseini et al.
The model parameter values are provided in Table 1. The parameters Mα ,
Mq , and Mη are unknown. We design the inputs such that the information con-
tent with respect to these parameter is maximized (see (6)). Other parameters
are considered to be known in this example. Significantly different initial pa-
rameter values (provided in Table 1) have been used to design the first optimal
input signal (prior to the experiment). Four parameter estimation and input de-
sign iterations (described in Section 3.3) are performed during the test run-time.
After each maneuver optimization, new parameter values are provided by the
EKF based on virtual measurement data generated from the model described
in (19)–(23). A sampling rate of 100 Hz is assumed and the run-time of each
maneuver is 4 s. The discrete time step for the solution of the optimal control
problem (6) with the direct method for optimal control is set to 0.05 s. We
perform 4 iterations of parameter estimation and input optimization during the
flight. The following input and state bounds are considered to design the inputs
η ∈ [−12.5◦ , 12.5◦ ] , (24)
◦ ◦
α ∈ [−10 , 10 ] , (25)
q ∈ −12◦ s−1 , 12◦ s−1 .
(26)
It can be seen in Fig. 2 that the structure of the optimal inputs remain
roughly consistent as the parameter values get updated. This is in agreement
with using the optimal inputs from the previous set of parameters as an initial
guess for the maneuver optimization in the next step. Figure 3 shows the outputs
of the dynamic system corresponding to the inputs of Fig. 2 at each iteration.
The change of the parameters and the respective estimate for their standard
deviations after each iteration are shown in Fig. 4. It can be seen that the
final values of the standard deviations in Fig. 4 are consistently lower than the
Cramér-Rao bounds in Table 1. This is due to the fact that the Kalman filter
uses the available data in all of the five maneuvers, whereas only one maneuver
run has been considered in Table 1. It is also worth mentioning that no process
noise has been considered in this study. The initial state covariance matrix for
each EKF run is set to
0.2 0.0
P0 = . (27)
0.0 0.1
The achieved parameter Cramér-Rao bounds are consistently lower than the
values determined for the same model in previous studies such as [5, 11]. This
is mostly due to the higher sampling rate in this study and the fact that only
three (main) model parameters have been selected for parameter estimation and
therefore also optimal input design. The results show that the proposed method
in this study achieves a similar accuracy in the parameter estimates to methods
that assume a more accurate initial guess for designing the optimal inputs.
Online parameter estimation and optimal input design 135
Table 1. Parameter values and optimization results for the linear short period
example.
True Initial Final
Cramér-Rao
Parameter parameter parameter parameter
bounds
value value value
Zα −0.737 −0.737 −0.737 −
Mα −0.562 −0.112 −0.543 0.0244
Mq −1.588 −0.318 −1.621 0.0335
Zη 0.005 0.005 0.005 −
Mη −1.66 −2.324 −1.687 0.0194
10
0
η [◦ ]
Offline
−10 iter 1
iter 2
iter 3
iter 4
−20
0 0.5 1 1.5 2 2.5 3 3.5 4
t [s]
Fig. 2. Optimal elevator deflection.
4.1 Parameter Estimation
Two parameter estimation experiments are performed to investigate the im-
provement in the accuracy of the parameter estimates if the optimal inputs are
adjusted during the experiment run-time, as suggested in this study. The param-
eter estimation experiments are performed using a special implementation of the
Maximum Likelihood estimation method in the output error form as described
in [3]. The same initial guesses are set for parameters as in Table 1. The results
of the parameter estimation experiment are provided in Table 2. It can be seen
that the optimal input adapted during the runtime of the experiment results in a
significant improvement in parameter accuracy. Figure 5 visualizes the absolute
value of the correlation matrices after each of the parameter estimation experi-
ments. The correlation between the parameters Mq and Mη is higher when the
adjusted inputs after optimization are in use. This may be contributed by the
fact that only the diagonal elements of the dispersion matrix D are explicitly
considered in the cost function.
136 B. Hosseini et al.
0
α [◦ ] −5
−10
0 0.5 1 1.5 2 2.5 3 3.5 4
10
q [◦ s−1 ]
0
−10
0 0.5 1 1.5 2 2.5 3 3.5 4
t [s]
Fig. 3. Response of the linear short period model to the optimal inputs with the
available set of parameters at each iteration.
Table 2. Estimated parameter values and their standard deviations (Std Devs)
for the linear short period example.
Estimated Estimated
Parameter Parameter
parameter parameter
std devs with std devs with
Parameter with the with the
the pre-flight the adapted
pre-flight adapted
optimal input input
optimal input input
Mα (−0.562) −0.61 0.0947 −0.562 0.0255
Mq (−1.588) −1.585 0.0585 −1.603 0.0347
Mη (−1.66) −1.688 0.0273 −1.668 0.02
5 Conclusion and Outlook
In this study we show that adaption of optimal inputs for parameter estimation
during the experiment run-time significantly improves the results of a subse-
quent parameter estimation. However, this is a very complex process and can be
difficult to successfully implement and apply to a real aircraft. First and fore-
most, the application of the proposed method in real-world systems is limited
by the existence of process noise and deficiencies regarding the model structure,
such as unmodeled nonlinearities. Among the implementation aspects, the low
computing power and memory capabilities of common flight control computers
can be considered challenges associated with this approach.
Furthermore, since the parameter values used to design the optimal inputs are
not accurate, state bounds are not guaranteed to be satisfied during the flight.
Online parameter estimation and optimal input design 137
0
Estimated Parameter 0.2 Parameter
−0.2 True Parameter Value 0.15 Standard Deviation
σ(Mα )
Mα
0.1
−0.4
0.05
−0.6 0
0 1 2 3 4 5 1 2 3 4 5
−2
·10
8
−0.5
6
σ(Mq )
Mq
−1 4
2
−1.5
0
0 1 2 3 4 5 1 2 3 4 5
−2
·10
4
−1.6
−1.8 3
σ(Mη )
Mη
−2 2
−2.2
1
0 1 2 3 4 5 1 2 3 4 5
iter iter
Fig. 4. Parameter values and their standard deviations at each iteration.
(a) Offline Optimization (b) Onboard Optimization
Fig. 5. Absolute value of the correlation matrices.
138 B. Hosseini et al.
To address this issue, more conservative state bounds can be considered for the
first maneuvers and the bounds can be expanded as more accurate parameter
estimates become available.
The additional efforts required to implement this method can however be
advantageous in use cases where flight test time is very expensive and no good
initial guess for the system model exists. In the case of linear system identifica-
tion, changing the trim point of the aircraft during the flight also changes the
values of the linear system’s matrix coefficients. This method can be applied to
adapt the optimal inputs of the aircraft to the new trim point during the flight.
It might be possible, especially in the case of nonlinear systems, that signifi-
cantly different parameter values result in significantly different maneuvers and
therefore cause convergence problems in the short time between two maneuvers.
This can be solved by updating the maneuvers online, as new parameter values
become available. Online methods for optimal input design are currently being
investigated by authors. Furthermore, it is planned to test the algorithm and
its capabilities in a real-world experiment using an unmanned aircraft. Utiliza-
tion of other online parameter estimation algorithms (e.g. the frequency domain
method as discussed in [10]) will also be investigated in future work.
6 Acknowledgement
This work is supported by the German Federal Ministry for Economic Affairs
and Energy as part of the LuFo program (grant-ID 20Q1719D).
References
1. Betts, J.T.: Practical methods for optimal control and estimation using nonlinear
programming. The Boeing Company, Seattle, Washington (2010)
2. Chen, R.T.N.: Input design for aircraft parameter identification: using time optimal
control formulation. AGARD CP-172 (1974)
3. Göttlicher, C.: System identification for aerial applications using optimal control
methods. Ph.D. thesis, Chair of Flight System Dynamics, Technical University of
Munich (2018)
Online parameter estimation and optimal input design 139
4. Gros, S.: Dual-mode batch-to-batch optimization as a Markov decision pro-
cess. Industrial & Engineering Chemistry Research 58(30), 13780–13791 (2019).
https://doi.org/10.1021/acs.iecr.8b06471
5. Hosseini, B., Botkin, N., Diepolder, J., Holzapfel, F.: Robust optimal in-
put design for flight vehicle system identification. In: AIAA SciTech, Or-
lando. American Institute of Aeronatics and Astronautics, p. 0290 (2020).
https://doi.org/10.2514/6.2020-0290
6. Hosseini, B., Göttlicher, C., Holzapfel, F.: Optimal input design for flight vehicle
system identification via dynamic programming and the direct method for optimal
control. In: AIAA SciTech Forum, San Diego. American Institute of Aeronatics
and Astronautics, p. 1319 (2019). https://doi.org/10.2514/6.2019-1319
7. Jategaonkar, R.V.: Flight vehicle system identification: a time domain method-
ology. American Institute of Aeronautics and Astronautics, Inc., Braunschweig
(2006)
8. Królikowski, A.: Application of input-signal design in system identification for
adaptive control. International Journal of Systems Science 17(2), 305–318 (1986).
https://doi.org/10.1080/00207728608926805
9. Mehra, R.K.: Optimal inputs for linear system identification. IEEE
Transactions on Automatic Control AC-19(3), 192–200 (1974).
https://doi.org/10.1109/TAC.1974.1100554
10. Morelli, E.: Autonomous real-time global aerodynamic modeling in the frequency
domain. In: AIAA SciTech, Orlando. American Institute of Aeronatics and Astro-
nautics, p. 0761 (2020). https://doi.org/10.2514/6.2020-0761
11. Morelli, E.A.: Practical input optimization for aircraft parameter estimation ex-
periments. ScD Dissertation, The George Washington Univeristy, Joint Institute
for Advancement of Flight Sciences, Hampton, VA (1990)
12. Morelli, E.A., Klein, V.: Aircraft system identification: theory and practice, 2nd
edition. Sunflyte Enterprises, Williamsburg, Virginia (2016)
13. Rieck, M., Bittner, M., Grüter, B., Diepolder, J., Piprek, P.: Falcon.m user guide
(2018), www.falcon-m.com