Hi Dhanunjaya,

if i interpret your grid search right, you plan ti use the k top rated features to distinguish patients and controls and choose the k which produced the best results? If yes, you could do so. However, two remarks:

1. there should be some data which is not invoked in the whole process (filter and the grid search) in order to test your method afterwards.

2. If your goal finally is classification you should think of a wrapper method (e.g. sequential forward floating selection) which takes into account your specific classifier and the relation between features

Greetings, Sebastian

Hello @Sebastian,

Thanks for your reply. That's what i think initially but i don' know whether grid search works or not to select the k value. I didn't think of wrapper method directly as i have total 1.6 million features out of that i need to select the k features. Is there any proof that how to select k number???

Hi,

i am not aware of a real good way to choose the right number k, apart from what you planned to do (if you have in mind to have separate data).

In fact, concerning the big number of features probably the wrapper won't provide good results and is not applicable, respectively. But also concerning this big number, have you that to apply a conventional dimension reduction technique as PCA?

Greetings, Sebastian

Hello @Sebastian,

I shouldn't use feature reduction technique such as PCA. Since my main task is to do the feature selection and take main features apart i.e., features responsible for the disease.

You could do PCA anyway and see afterwards if there are specific features which closely belong to single PCs.

By the way: considering the big number of features you will have large redundancy and reduncdancy is likely to affect the filter for feature selection as well. I.e., apart from not knowing the number k to choose, i would not be sure if you can trust your list.

Greetings, Sebastian

Dear Dhanunjaya Mitta

I'm very interested in your high-dimensional data set.

Can you share the data set with me? Recently, I revised my feature selection method, BSSReduce. The new version may 10 times faster than the original one.

Thank you!