For my point of view, I would recommend a hybrid PCA-LDA to reduce large size of Datasets. I have had Dataset size 18 Gigabyte with positive results for the Data reduction.

Regards

Aws

Apply dimensional reduction algorithm such as PCA, NNMF. You can look at the variance of their component set up a threshold values and extract features only with higher cumulative variance (90%). Another methodology is to apply genetic algorithm which is also used as dimensional reduction technique but you may need to model you data with a fitness function. Regression models can also be used for feature space selection and use different criteria such as AIC or BIC and select features based on those.

Some times it is necesary to reduce the number of variables through a PCA

Dear Dr. Daqing Chen ,

I suggest you to apply feature selection before applying classification algorithms. Also, Nature inspired algorithms such as PSO, firefly , cuckoo search and etc... are very useful for high dimensional datasets.

You can use PCA for dimension reduction. But if you don't want to change the original features, it's better to use feature selection approaches such as Wrapper and Filter. Here is a paper link which can be useful:

https://ieeexplore.ieee.org/document/7745366

You could use embeddings: https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526

There are few steps that I feel good so:

1. Compute correlated dimensions having classes and choose one among them.

2. Consider dimensions which explains most of the variance in dataset.

3. Try to club two or three classes in one class and by this, you would have fewer classes instead of bulk of classes.

4. Ask business which features they feel are more important to them to be considered.

random forest may be a good solution and its really depends on howyour research field

Can be used

kernel PCA

1- Input points before kernel PCA

2- Output after kernel PCA with k(x,y)=(xTy+1)

or

cuckoo search

The fundamental reason why items need classification is that the treatments to the different classes are different. Using this principle, you can explore avenues to rationalize the number of dimensions to a more manageable value.

Also, partitioning the problem into a main-classification (maintained in database) and sub-classification (performed live on filtered data) might help. Such a model can be leveraged to club dimensions which can be separated later.

Try Latent Dirichlet allocation, Topic model

No one mentioned this yet..

You could also use Factor Analysis for dimensionality reduction. I believe it's more powerful than PCA, but it also requires more human involvement in interpreting the results and selecting the appropriate attribute subsets.

Here's a nice discussion on the differences between PCA and FA: https://stats.stackexchange.com/questions/1576/what-are-the-differences-between-factor-analysis-and-principal-component-analysi

We use Ivis for non-linear dimensionality reduction and feature extraction (https://github.com/beringresearch/ivis). We have successfully applied it to real-world datasets with as many as 50k features. Ivis implements a standard sklearn-friendly API and can be easily integrated into an ML pipeline. Paper describing the algorithm is here: https://www.nature.com/articles/s41598-019-45301-0

Dimension Reduction

One of the best way to reduce the dimension is Feature Selection (FS). It is a process to select a near-optimal subset of features within a reasonable time stamp. Thus it improves the speed of classification (due to lesser feature dimension) and increases the classification accuracy by removing redundant features. Metaheuristic algorithms like Genetic Algorithm, Ant Colony Optimization etc. can be used to suit the purpose.