Hi everyone!
I analyzed 20 tissue samples of oral leukoplakia (OL - an oral potentially malignant disease) through untargeted metabolomics to compare the metabolic profile of those OL who had malignant transformation (5) and those who did not (15). I know that the small sample size is one important limitation of the study, but OL is a rare disease and I have to deal with it.
Well, when I use my complete dataset (around 4k compounds) to perform multivariate analysis such as PLS-DA, my model is overfitted, exhibiting a negative q2. However, when I use the 72 compounds considered statistically significant by the univariate methods (hypothesis tests) as the input data, my q2 rises to 0.6. The improvement also occurs when I use this small dataset to build the heatmap that clearly distinguishes the malignant transformed from the non-transformed OL. Interestingly most of the compounds classified on the PLS-DA VIP list are the same, both using my whole data and using the 72 discriminant features as the input.
I recently presented my thesis to a metabolomics specialist and she told me that my analysis is curious and that she cannot tell me whether it is right or wrong.
Would anyone here help me with this question?
Thanks!
Hi, in my opinion you can build a PLS model using 'discriminant' features, but the outcomes (predictive performance, etc) from that model will not be generalizable and overly optimistic as you might get a similar result using random labels. To avoid overfitting the feature selection should only be applied to the training set (so you would need a test test). I would suggest instead to use a model that allows adding a regularization term into a cost function. This way you can control the effect of uninformative features by minimizing their weights.
HI, you are actually asking two different questions here, as PCA is only a data reduction method while PLS and clusterization aim to interpret the data.
You have 2 groups and only 20 samples all in all, so of course way too many compounds and potential confusions. And I would not rely exclusively on "significantly different" between the 2 groups to select data as, with 4k variables, it is more likely than not that you may have false positive (depending on how stringent you ran the statistical test).
My approach would be to do the PCA and look, is there some spontaneous separation of the 2 groups on one of the early principal compoments? Do your data "cluster" i.e. do you have groups of very correlated data? In which case you can probably simplify these groups to a single variable.
You have only 20 samples so when there are significant differences (72 variables) it might be worthwhile to actually plot and look, is the difference pulled by a few samples or is nicely repeatable in one group.
Anyway you look at it, you are in trouble... too many variables for few samples, this is going to require a lot of brain exercise. This is where statistical tools, however good they are, must leave way to knowledge and human intelligence. Do you have hypotheses as to the related biological mechanisms to help you sift through the data , and the predictive value, as said by Guillermo Quintas, will stay poor. Tentative I'd say, it might be a tool to help the practician but not totally relaible as a diagnostic.
Hi Roberta,
I guess you want to find the discriminative or important variables from your dataset, don't you?
Normally, we can do PCA analysis for that kind of fat dataset (variables >> samples) in the first place. It is useful to understand the natural separation in your dataset. Also, you can check the loading of the PCA, what variables that significantly drive such separation.
For further analysis PLS-DA may help a lot to find your VIPs from each sample class. However, considering your 4K variables, it might be a good option to compare several machine learning feature selection algorithms to find a good model as well as to determine your important features to get more reliable data.
Good luck!
- Badges
- Science method