As the models used in Kalman filtering are also Gaussian processes, one would expect that there would be a connection between GP regression and Kalman filtering. If so, could one claim that GP regression is more general than Kalman filtering? In that case, aside from computational efficiency, what would be the other advantages of using Kalman filtering?
See, e.g.,
http://www.cs.washington.edu/robotics/postscripts/gp-ukf-iros-07.pdf
http://arxiv.org/pdf/1504.05994.pdf
Cheers, Rolf
GP regression, as the name suggests, is a regression method, and provides a flexible parameterization for mappings which may not otherwise be captured in closed-form. 'Gaussian process' simply refers to the particular structure of the parameterization, which uses Gaussian conditional expectation; see http://www.robots.ox.ac.uk/~mebden/reports/GPtutorial.pdf, for example. On the other hand, Kalman filter provides a method of estimating conditional distribution of a 'hidden' or latent Gaussian variable recursively, as measurements become available on another jointly Gaussian variable, by combining model predictions and the measurements. A lot is known about the asymptotic (stability) properties of the KF, which allows us to 'hard-code' its time invariant/limiting form in engineering systems.
In short: KF and GPR are different beasts. As Rolf has pointed out above, GPR can be useful in certain extensions of KF to nonlinear systems.
Hope this helps.
Paresh
Thank you Rolf and Paresh for your useful comments.
I read the references that you have cited. However, I am still not sure if these references fully answer my questions. So let me rephrase my questions.
Kalman filter recursively produces estimates of unknown variables based on system’s dynamics model, known control inputs to the system and multiple sequential measurements.
In Gaussian process regression, same process can be implemented in order to estimate the unknown variables. The system dynamics can be implemented in the form of Gaussian kernel and hence incorporated in the covariance matrix. The inputs and sequential measurements are then through use of the Bayes rule can be used to estimate the unknown variables.
My question is the following. Is there any problem that can be solved by kalman filtering which cannot be solved by Gaussian process regression? In this relation, I am not thinking about computational efficiency but on if, kalman filtering can offer insights, which are not already offered by the Gaussian process regression.
Dear Ali,
You will find a full detailed answer to your question by following this related question on ResearchGate (in particular by checking out Least-squares estimation: from Gauss to Kalman) :
https://www.researchgate.net/post/What_are_other_modern_alternatives_to_Markov_Chain_models
Regards
Please, see Section 10 in
L. Martino, J. Read, "Joint introduction to Gaussian Processes and Relevance Vector Machines with Connections to Kalman filtering and other Kernel Smoothers", 2020.
at
https://vixra.org/pdf/2009.0135v1.pdf
(it will be in arXiv next week)
and other references for instance
S. Sarkka and J. Hartikainen. Infinite-dimensional Kalman filtering approach to spatio- temporal Gaussian Process regression. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, volume 22 of Proceedings of Machine Learning Research, pages 993–1001. PMLR, 21-23 Apr 2012.
A GPR run in some time ordering should correspond exactly some some filtering problem with the same covariance structure in the time evolution which I suppose could be an SPDE which might make the analysis a bit more complicated.
- Badges
- Science topic
- Similar topics
- Statistics
- Time Series