Hello, I am working on a model:
dx/dt = Ax + Bu
y = Cx + e
It is equivalent to an output-error model where noise is not modeled. I have trained it using real data through maximum likelihood estimation (MLE) and parameters in A, B, C are estimated.
This problem is that the residual is not white. Will this violate the assumption of MLE that y is a sequence of independent variables?
Then you have to deal with some bias. How severe this is, depends on your model assumptions and data.
If I understand this correctly, in your model you assume the distribution of Y with a density of f and build log-likelihood based on the independence assumption, which is probably a product of f from i=1 to N (number of your Y samples).
Assuming Y_1,...,Y_N are not independently distributed, then the likelihood function would not be a mere product. If not corrected, you will have to deal with bias from model-error. However, the bias is small when the Y-dependence is also small. In general, model-error bias does not converge to zero when n goes to infinity. So either correct your likelihood or choose other methods.
Christopher Paulus Imanto ,
Thanks for your answer. You are right. My likelihood function is mere a product of all probability density functions. Since the distribution of noise is prior unknown, I may not be able to correct my likelihood function. Even if using different method, e.g., a quadratic formulation, I am afraid there will still be a bias in model-error. However, I found in a textbook saying:
However, as long as the bandwidth of the noise spectrum is much larger than the system’s bandwidth, the noise can be seen as practically ‘white’ (= i.i.d Gaussian since the noise is normally distributed).
APA (American Psychological Assoc.) Raol, J. R., Singh, J., Girija, G., & Institution of Electrical, E. (2004). Modelling and Parameter Estimation of Dynamic Systems. London: The Institution of Engineering and Technology.
Is this the same as your saying "the bias is small when the Y-dependence is also small"?
Regards,
If the residual is not white. This really violates the assumption of MLE that y is a sequence of independent variables.
You can separate colored noises into a colored part and a white part, and put the colored noise part into the system model, and live the white noise alone. it may solve the problem.
regarding your question Zequn Wang,
Without reading the paper you gave, I suppose it's not the same (in general). The bandwidth does not necessarily have anything to do with system's dependence. But they may refer to the same construction, which you study.
I can explain it more, so that you can decide yourself, if they contain the same information. To make this easier let assume we only have two r.v: Y1,Y2. If independent, then f,ind_(Y1,Y2) is equal to f_(Y1)*f_(Y2). Otherwise it would deviates from f,ind_(Y1,Y2). In fact f,dep_(Y1,Y2)=f_(Y1|Y2)*f_(Y2). So the difference between f_(Y1|Y2) and f_(Y1) decides the magnitude of your bias, since this is the only difference coming from the choice of density. If they are both very similar (in other words Y2 does not give a significant information to the Y1-distribution), then we are dealing with very similar densities (with independent assumptions or not).
I would not call the difference between f_(Y1|Y2) and f_(Y1) a "noise", since this is technically not a (random) noise. This also does not decide anything about bandwidth of the distribution you are working at.
hope it helps.
Christopher Paulus Imanto
Thanks a lot. Your explanation is very clear to me now.
- Badges
- Science topic