Pekka already answered your question to me.

Naturally both your development data and validation data require pre-processing. What I mean is that you should under no circumstance pool both data sets and pre-process it as a whole for classification. Take for instance normalisation as a pre-processing step. Development data is used to determine scale and shift parameters. Both the development and validation data are then normalised using these parameters. If we were to use the entire combined data set for normalisation, we would bias our classification models. The other alternative (a separate normalisation based on the validation set) is not as problematic, but may decrease classification performance if the validation set is small. This does not only apply to normalisation, but to other pre-processing steps (imputation, PCA, clustering etc.) as well.

Thus, you may perform one analysis on the combined data set, but the information generated should generally not be used for building models. In your case, you may perform a segmentation study on the whole data set, and also perform classification if, and only if, segmentation is not population-based (i.e. no information about other data is used when segmenting one data set), or if the segmentation results are not used for classification.

You can use unsupervised machine learning method for quality assessment of tandem mass spectra without any training dataset.

Hi, in unsupervised discretization you do not use class labels and hence 1) with k-means clustering (no labels) you either use unsupervised discretization or you do not discretize attributes. Regarding the more tricky question 2), I would not apply supervised discretization afterwards because it changes the kmeans based attiribute patterns.

P

In general, I would be very careful when pre-processing data with the aim of generating a model. When modelling, it is important to keep development data, used to train a model (whether supervised or not), separate from validation data, that is used to evaluate model performance. Otherwise, biases may be incurred as information concerning validation data is incorporated into a model.

Therefore, one should perform pre-processing on the development data only, and subsequently transfer and apply the pre-processing results (scale and shift parameters for standardisation, clusters formed, imputation models, etc.) to the validation data set.

Other then that I don't think it is necessary to specifically use either supervised or unsupervised methods for pre-processing. Using supervised methods may however increase the risk of overfitting, as pre-processing will be somewhat adapted to the labels of the development set. Using unsupervised methods this risk should be somewhat less, though naturally you will also learn some of the patterns specific to the development data as well.

@Sanket Suthar, thank you for your reply regarding the application of unsupervised machine learning on examining the quality of training dataset.

@Pekka Jounela, thank you for your feedback. I totally agree with you regarding the need to remove the dependent variable before implementing the k-means clustering analysis. In this case, I am performing different analyses; I am not doing a cluster-based classification model. So, the first part of the study is to perform a segmentation studies on the biomedical data set, while the second part of the study is to perform a build a classification or predictive model based on the biomedical data set. Sorry for not clarifying clearly in the question. In this case, can I perform different pre-processing method, that is unsupervised and supervised discretization, on the the same dataset for two different purposes.

@Alex Zwanenburg, thank you for your advice.I do not understand your phrase "... transfer and apply the pre-processing results ...to the validation data set". Do you mean that we do no need to apply the same pre-processing method on the validation data set that will be used as an input to the developed classification model (during the validation phase) and not the cluster model? It sounds like I am asking two different questions in the above sentence, but please could you clarify and advise me accordingly.

Kevin, thanks for the clarification of the second task. In supervised classification just remember to use the same pre-processing (discretization/normalization etc.) model separately for testing/cross-validation examples. What I mean is explained in this blog (see Figure 2.): https://rapidminer.com/learn-right-way-validate-models-part-4-accidental-contamination/

...and a good old paper to read:

Dougherty J, Kohavi R, Sahami M. Supervised and unsupervised discretization of continuous features. In: Proceedings of the 12th

international conference on machine learning. San Francisco: Morgan

Kaufmann; 1995. p. 194–202.

Pekka already answered your question to me.

Naturally both your development data and validation data require pre-processing. What I mean is that you should under no circumstance pool both data sets and pre-process it as a whole for classification. Take for instance normalisation as a pre-processing step. Development data is used to determine scale and shift parameters. Both the development and validation data are then normalised using these parameters. If we were to use the entire combined data set for normalisation, we would bias our classification models. The other alternative (a separate normalisation based on the validation set) is not as problematic, but may decrease classification performance if the validation set is small. This does not only apply to normalisation, but to other pre-processing steps (imputation, PCA, clustering etc.) as well.

Thus, you may perform one analysis on the combined data set, but the information generated should generally not be used for building models. In your case, you may perform a segmentation study on the whole data set, and also perform classification if, and only if, segmentation is not population-based (i.e. no information about other data is used when segmenting one data set), or if the segmentation results are not used for classification.