If I want to build a time series model, of which the output and the input are both multivariate, I need to do a feature/variable selection work. How can I measure the input vector and the output vector? Does the mutual information work?
The multivariate mutual information can be positive or negative, I think it is not easy to compare that, right? Plz share your opinions.
This is the link from wiki about MMI:
https://en.wikipedia.org/wiki/Multivariate_mutual_information
Do you mean you are going to build a multi-dimensional model like the coupled map lattice (CML) models. There are essentially not difference between the modelling of a NARMAX model or a multi-dimensional model, such as CML. In this sense, all the methods which works for the modelling of time series also works for modelling of multi-dimensional systems. Actually, there are plenty of papers which published in the modelling of spatio-temporal systems you can refer to, for either cellular automata or coupled map lattice systems, and even the partial differential equation models. Please search for the papers written by S. A. Billings, D. Coca, Y. X. Yang, Y. Pan, Y.F. Zhao and me as well, between 2000 and 2010. Yes, you can use mutual information as the criterion for the variable selection however the calculation of the MI for a continuous-state signals (has infinite number of states) could be difficult which highly depends on how to box the states or a kernel based method to estimate the distributions. This is greatly different from the finite-state cases such as cellular automata. Hope this helps.
Yuzhu
Do you mean you are going to build a multi-dimensional model like the coupled map lattice (CML) models. There are essentially not difference between the modelling of a NARMAX model or a multi-dimensional model, such as CML. In this sense, all the methods which works for the modelling of time series also works for modelling of multi-dimensional systems. Actually, there are plenty of papers which published in the modelling of spatio-temporal systems you can refer to, for either cellular automata or coupled map lattice systems, and even the partial differential equation models. Please search for the papers written by S. A. Billings, D. Coca, Y. X. Yang, Y. Pan, Y.F. Zhao and me as well, between 2000 and 2010. Yes, you can use mutual information as the criterion for the variable selection however the calculation of the MI for a continuous-state signals (has infinite number of states) could be difficult which highly depends on how to box the states or a kernel based method to estimate the distributions. This is greatly different from the finite-state cases such as cellular automata. Hope this helps.
Yuzhu
In fact i am following your papers closely, Dr. Guo. I have got some idea from the papers listed below that MI can do measure the relationship between two continuous vectors. They use the kNN to calculate the MI. By the way, the aim I want to achieve is to get a compact form of regressors matrix to avoid the curse of dimension, so first I will select the most relevant regressors from the orginal ones. Please share your opinions.
Here is the paper title "Estimating Mutual Information"
http://arxiv.org/pdf/cond-mat/0305641
In probability theory, mutual information measures the dependece between two random variables. These two vectors could be the observations of an event. You could use this metric, but you also could use Correlation metric to measure the dependece between these two vectors.
Dear Lázaro , Correlation can not reflect the nonlinear relationship between two random variables. I wanna to check whether the MI could measure the dependence between two variables that both are vectors with real numbers. What about one vector with real numbers and the other with discrete numbers?
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering