In machine learning and statistics, dimensionality reduction (DR) is a fundamental technique of revealing the intrinsic low-dimension features hidden in a high-dimesnsion dataset. There are various DR algorithms, linear ones, like PCA, MDS, etc, and nonlinear ones, like ISOMAP, LEE, etc. The task of DR actually is to implement a mapping that transforms a set of high-dimension data to a set of low-dimension data. What I have concerns about DR is how to use the low-dimension dataset. Can the obtained low-dimension dataset be used for other machine learning tasks, such as classification or clustering analysis? Could someone give me some ideas or references. Thanks in advance.
Ziyan, thank you very much for your answer. I know DR implements the mapping X={x1, x2,..,xn}-->Y={y1,y2,...,yn}，where xi and yi are of dimension D and d respectively, and d
Dimensionality reduction has potential applications of as bellow, as you can explore.
1. It reduces the time and storage space required.
2. Removal of multi-collinearity improves the performance of the machine learning model.
3. It becomes easier to visualize the data when reduced to very low dimensions such as 2D or 3D. Regards.
Thanks
Regards,
M M Manjurul Islam
Yes. I think that the low-dimension dataset may be be used for other machine learning tasks.
Most of statistical techniques make use of the matrix X^T X, that is the dot product matrix, that is equivalent to the distance matrix. For instance, linear regression, but many others. If your dimension reduction technique preserves distances (or try to preserve) then you can apply virtually any statistical procedure on the projected data.
Yes the obtained data in the new feature space can be used for clustering or classification. In fact, in the new representation of data you will look for the most pertinent components and apply for example clustering on it.
Of course, the data set in low dimension can be used in classification algorithms such as Artificial Neural Networks (ANN), Random Forest (RF), Vector Support Machines (SVM), K-Nearest Neighbors (KNN). But you should keep in mind the dimensionality reduction approach, here there are two: a) Feature Selection and b) Feature Extraction. The first maintains the nature of the original data (maintains the original attributes) and the second takes new values (generates new attributes). This is something that you should keep in mind according to the problem to be addressed in order to properly select the Automated Learning (ML) algorithm. I give you an example. I work in the diagnosis of faults in rotating machinery using ML algorithms, a process prior to the classification stage is the selection of relevant attributes, therefore I use the first approach to dimensionality reduction, since after applying, for For example, KNN and If I get good results, I can study the physical behavior of each attribute since it does not lose its nature.
The first chapter of the book Spectral "Feature Selection for Data Mining (Zheng Alan Zhao and Huan Liu)" gives you a good idea of these two approaches.
- Badges
- Science topic