When I change the seed of random noise generation in MATLAB, keeping everything else fixed, the performance of the EKF and also UKF (MSE, ACF of the estimation errors, etc.) changes every time. Why does that happen? s it because of suboptimality of the nonlinear algorithms? Even if that is true, why are they sensitive to noise?
I have also seen the effect in Linear KF, although it not that pronounced. Is there any explanation for this, or do we have to just live with it?
The state noise covariance matrix Q and measurement noise covariance matrix R should be consistent with the random noise. Everytime you change the random noise, you need to change the value of Q and R.
Hello,
What I meant is slightly different
I have fixed Q and R.
All I do is change the seed from which random numbers are generated - which are eventually multiplied with sqrt(covariance matrix) to get the random noise. So, the noise statistics remain same. Only, the seed from which MATLAB picks up random numbers changes every time.
All else is exactly the same.
Hope that clarifies
By EKF do you mean the Extended Kalman Filter? If you are using the EKF with a nonlinear model, if will be computationally unstable -- it will respond sensitively to the particular random numbers of each simulation, so it might run quite well with one seed, and quite poorly with another -- for the first few timesteps, anyway.
I think the best way to handle it is to present your results as averages over many realizations. This is pretty reflective of real life problems that are sensitive to initial conditions, so it makes sense to show average results. I have a paper where I did experiments with the EKF and a chaotic dynamical system (" Balance dynamics and gravity waves in four-dimensional data assimilation " - you can download it here on research gate) -- I had similar issues where the different experiments I ran had really different outcomes each time, so I looped over several hundred simulations and looked at the statistics of what came out.
Also, the Ensemble Kalman Filter is usually a lot more stable in a nonlinear model than the EKF.
Hello,
I'm afraid I don't understand clearly what you do mean by "sensitive to the noise". Does that mean the result diverge or simply the error changes with time? Perhaps you can exam the error covariance P with time. If the algorithm is unstable, the matrix P may reflex it. If the matrix P does't change significantly each time, your results may be credible, the reason is that the "measurements" may be not perfectly chosen.
I think that you program the code in m-file and generate the white noise by function "randn". The seed of "randn" is set ahead of calling the function. In this case, if the seed is unchanged, every time you call "randn" you will get the same return. It means that every time you change the seed, you run the EKF in only one condition. I suggest delete the "seed" and ran multiple times and see whether the result diverge.
As far as I know, many reasons may contribute to the result of a EKF, tuning the EKF is not a simple job. Above all you should exam the code carefully. If the linearity of your problem is really high, other nonlinear filtering algorithms may be better choice. Good luck!
I hope I have made myself understood. Thank you for your reading.
I had the same problem when applying the EKF, but the differences were not significant and the algorithm remained stable. When the problem is highly nonlinear, the sensitivity to the random numbers becomes more significant and it is better to use more stable algorithms.
Good Work.
For each different seed (without changing anything else) you get a different realization of a (pseudo)-random sequence; hence you get different results when running your EKF. Usually you would run many experiments (eg several hundreds) with different seeds and ensemble average the results, in order to have statistical information "independent" on a particular realization (this is often called a "Monte Carlo simulation").
It is the fact that the performance if the EKF depends upon the value of noise. To decrease the sensitivity, the value of Q has to cover the standard deviation of noise; by this is meant that the value of Q must be bigger.
I completely agree with Phillipe Martin. As you might know, Matlab is not able to provide a completely random signal. Instead, you need to live with the concept of pseudo-random signal. Then, answering your question directly, every time you change the seed of the noises what you produce is a "different" random signal and consequently a different response of the filter . What I usually do when working with EKF's and UKF's is letting these seeds random as well.
To get rid of these random ACF errors, one has to carry out a Monte Carlo set of seeds (approximately 100 even more) and averaging the ACF's or ARMSE's will result in consistent results. More over, fixing all other parameters and simply going with the noise is an innocuous practice. It is often recommended to go with uncertainty in initial conditions, state noise and measurement noise.
I agree with the last answers that suggest to follow a Monte Carlo approach. An example of this approach and all the necessary equations to analyse and represent the result can be found in the following paper, for both EKF and UKF.
Article How To NOT Make the Extended Kalman Filter Fail
- Badges
- Science topic