it is related to neural networks and basically i have to train the network. there are 14 electrodes and 70000 features.so ineed to reduce the feautures without losing information
You can apply on of the common techniques for dimensionality reduction. Have a look at singular value decomposition (SVD) or principal component analysis (PCA). The latter may need more time to be trained
You can easily experiment with these methods in python sklearn library. Both SVD and PCA reduce your input significantly provided that some of the features are correlated. They also allow to check the remaining variance in order to have an idea of how much information you loose.
Another idea is to reduce dimension with variational or neural network based autoencoder, which could be a little more tricky to optimize.
Let us know if you need more references or tools to be suggested. I would start with SVD and see how it goes
currently i am working in c++. so any suggestions on how to implement svd or pca in C++
You have a library for that, no need to implement https://code.google.com/archive/p/redsvd/
As a matter of advice, I'd still recommend debugging in sklearn, cause it is nice for analysis. If you reduce your feature vectors, nothing stops from dumping them to file and then run your C++ code on compressed features.
I have had some luck with random orthogonal projections for sparse data.
http://vision.fe.uni-lj.si/docs/danas/cvww10final.pdf
I used matlab to generate the random matrix in eq4 and plugged into eq 1. 2 lines of code. It worked very nicely for my application that had very sparse data. It is also very fast.
check out:
http://scikit-learn.org/stable/modules/unsupervised_reduction.html
for some code and other ideas.
Calculate the weight or sum or some measure or lenghth of a row and column if find that row or column not much effect on main result delete one row and one column at a time. That will reduce the dimension of the matrix.
Dear Sowmith,
If you find faith in our nested algorithm technology (a different set of coauthors) for information and financial processing (check my website on RG, especially the paper on math stat. pricing) you should divide your stack into groups of 7000 using some criteria and check whether repeated estimation gives stable results. Or else take averages. The proof of this is slightly complex but it exists. SKM, NH, SM
I tjhink that is not a big matrix 13*70000=980 KB this is a very little space for a PC with one GB. You can to manage this in C that is adaptable to almost any package.
I agree with Gorin because it is inline with the requirement of the question.
I am not familiar with the time requirements for working with neural nets as you mention, but in general, the size of the matrix can be substantial for efficient processing and aggregation/disaggregation techniques may be employed. The size can be quite consequential if doing, for example, integer or nonlinear optimization. Search Operations Research journal 1991 for a framework on agg/disagg.
Simple cluster analysis can be quite effective at determining which of the 70000 are indeed identical/close to each other, and that may be the "easiest" way. As previously mentioned, factor/principal component analysis can be used, but other methods such as discriminant analysis or other also exist. In general, I would think cluster analysis with Ward's method or Avg. Linkage would be indicated. How far do you need to cluster (level of clustering) is a question....50,000? 35,000? 10,000? .... In general more accuracy is lost the more you cluster.
If most of your matrix elements are zero, you may convert it into a sparse matrix.
Your new sparse matrix will have just one dimension with a number of elements equal to the number of non-zero elements in the original matrix.
To process sparse matrices, you should write your own algorithms in your desire programming language. You can find several examples in C and C++.
The answer by Ali Mansoor doesn't concern the reduction of the dimension, but how to store a sparse matrix in dense form. This was not the question.
Btw.: Which matrix do you want to look at?
Is it really 14 electrodes giving just one signal, i.e. a vector of dimension 14, which the network should use to identify 70000 features? Or is it a sequence of signals of dimension 14 over time? That would of course give much more information.
If the task really is to identify such big number of features on the basis of just 14 given values: Who determined that number 14 beforehand?
You want to reduce the number of features classified by the network - somehow you want to solve the problem of how the network should react on data before training it.
You could see this e.g. as a question of statistical analysis of the data (regarding the features): What are the covariances? Do there exist dependencies between the features in the real data?
It simply says that there are alternatives to using a neural network.
I do not know what a 14*70000 matrix is. However, Lanczos reduction methods are very good for keeping important parts of a matrix in greatly reduced sizes.
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Materials Science
- Plating