Hi guys,

Reading my question title surely has given at least some of you a flashback on their experiences during the estimation of a nonlinear system model. I hope to get some tips, tricks and useful critic on my proceedings with the model estimation. The project I’m working on is part of my bachelor’s thesis. I am thankful for every useful input form you!

What am I identifying?

My task is to identify a fully-fledged driving simulator capable of movements in nine degrees of freedom in total. The main goal is to obtain a good system model which fits the estimation and validation data and can be used for further investigation (if needed). Most importantly, as I’m conducting the thesis with an automotive OEM, not only the identification per se but also the whole process from generating measurement data, selecting a suitable model and optimizing the parameters shall be worked out in order to have a reference for future research purposes.

The driving simulator has been shown to by nonlinear and dynamic and shall be investigated as a MIMO system.

What has been done so far?

Measurement: All nine degrees of freedom have been excited with suitable position signals (Sine sweeps, discrete sine excitations, white and pink noise, Amplitude modulated pseudo binary signals) and the output has been measured as acceleration. Not only have the individual degrees of freedom (longitudinal, lateral, …) been excited (which would be a SISO case) but also a multidimensional excitation (by exciting all degrees of freedom) has been performed to identify the MIMO system.

Model selection: I am working with Matlab’s built-in toolbox from Prof. Ljung as well as a Lolimot identification toolbox from Prof. Nelles. So far, I have gained deep insight in both toolboxes and examined the different approaches in more or less full detail. In the beginning, I have played around with the GUI to get a feeling for the system models. Now, I’m exclusively working with the toolbox functions in Matlab scripts to change the model and estimation parameters arbitrarily. I want to concentrate my thesis on the estimation of a Lolimot, Narx, Hammerstein-Wiener and a linear model. This way, I want to compare the different models and I want to show that a linear model for example is not sufficient for the underlying driving simulator. In conclusion, I want to find the model that performs best for my system.

What am I planning to do next?

In the next steps of my bachelor’s thesis, I want to examine the above mentioned system models and thus have to perform a parameter optimization. The models rely on a different set of parameters (e.g. time delays, nonlinearity estimator parameters, …). As testing out all parameter combinations does not seem to be a viable option w.r.t. computing time, I have defined a DoE and want to perform a subset selection which will be representative of all parameter combinations. Using this subset (which is noticeably smaller than the huge amount of parameter possibilities of the DoE) the models shall be estimated and compared using their respective loss function values. This allows me to assign a unique value to every parameter set of the obtained subset which reflects whether the model is better or worse. Next, I want to build a response surface model and find its global minimum to find the best parameter combination of the whole subset and consequently of all parameter variations.

What questions do I have?

Before I work on the above mentioned parameter optimization, I want to make sure that I have understood everything this far and that my data is suitable for an identification. I have gained quite some understanding reading various system identification publications, however I still am not sure on two things.

Excitation signals:

The above mentioned excitations have been measured with a set of acceleration sensors all around the vehicle mockup. The measurement output has shown some pretty good results, which I used to identify other system properties like latency, phase lag, etc. I am sure that the measured signals themselves are pretty good and show minor noise in the relevant frequencies and obviously a bit more noise for lower frequencies where the noise characteristics of the sensor itself takes over. However, I am not sure whether the type of excitation is right. For dynamic systems sine sweeps and APRBS signals have yielded good results in the literature. However, an APRBS signal (step excitations with different amplitudes) shows steep peaks in the measured output of the simulator. The vehicle moves (for vertical signals) up, idles a few seconds and moves down again. The peaks result in the steep movement up and back down again. Between that is just dead time. Thus, I am not sure, whether the system dynamic has been excited strong enough. A sine sweep seems to be better and the system models estimated with both toolboxes seem to confirm that or at least manage to obtain a fit to the estimation sweep data, whereas the APRBS data is very hard to fit.

So the question here is: Is such an excitation with dead time between measurement output peaks even suitable for an identification?

Another question is: The discrete sinusoidal excitations have been designed to excite the system with one sinusoidal signal which is faded in and out, then there is 2 seconds of dead time and then the next sinusoid follows. The measured output follows suit and shows excitation with dead time between the sinusoids. Is this critical as well?

The final question here is: I have also conducted measurements with white and pink noise inputs. The statistical character makes this kind of input especially useful. Though, the signals had to be manipulated in amplitude and smoothened to not overexcite the simulator dynamics (and eventually to crash the simulator). This means, that the frequency band is not as wide as a ‘normal’ white noise, but should be in the relevant are of the simulator. Is an identification with that kind of estimation signal suitable?

Estimation and validation data:

When estimating the system models, estimation, validation and test data can be assigned. The system is being estimated based on the estimation data (training data) and can be validated by plotting the system output for the validation data. What I fundamentally do not seem to understand, or have not read yet, is whether the estimation and validation data can be fundamentally different. In most examples I have seen that the system has been trained with e.g. step inputs and validated with a different independent set of step inputs. It was then tested again with a third independent step input. What I am trying to do however, is to estimate the system based on e.g. the Sweep data, to and to validate it on the white noise signals. The question thus is: Is that even a good approach? The signals are fundamentally different.

As far as I understand or want to understand is that a successful identification of a system should be capable to represent all input-output combinations possible for the system. It is very clear to me that this will never be the case. But the underlying system in my case is able to perform sine and step excitations and many more. Should I have measured an input-output combination that contained all kinds of excitations?

In other words: What is the best way to estimate and validate the model in my case? Ljung’s toolbox does not even take validation data in consideration during estimation, it much rather relies on the user to evaluate the fit to the validation data. This is very understandable, since in most cases the evaluation is a mere decision of the user.

I am thanking all of you for input to my problems!