Parminder,

Please explain a little what is originally the problem is. There several methods of reducing dimensions other than PCA/FA which can be theoretically and numerically constructed.

In a recent paper (Toutounji and Pipa 2014), we used the the algorithm and code by Kraskov A, Stogbauer H, Grassberger P (2004). Mutual information on multidimensional sources is computed by an adaptive k-nearest-neighbor estimate of probability density. Our sources where 9D input sequences to a recurrent binary neural network and the response of the 100 neurons comprising the network. Given the high dimensionality of the recurrent neural network, one needs many samples to have an accurate estimate, which results in a huge computation time. It was thus necessary to use PCA on the synthetic data and choose only those components that covered 95% of the information.

In the attached link, you can download the executable binaries, the source C code, and MATLAB rappers, all made available by the authors.

I hope that helps.

http://www.ucl.ac.uk/ion/departments/sobell/Research/RLemon/MILCA/MILCA

Article Estimating Mutual Information

Article Spatiotemporal Computations of an Excitable and Plastic Brai...

Excellent Dear Hazem Toutounji, I agree with your arguments and thanks for great sharing.

You can find some estimators in the ITE toolbox: https://bitbucket.org/szzoli/ite/