My lab has recently generated some medium size data sets that describe gene expression (qRT-PCR data) as "Delta-Ct" against patients. As you'd expect, the data set isn't complete - so for some patients IL-2 (say) might be undetectable, so there is no numeric value for the Delta Ct. Also, the patients fall into more than one category - sick, healthy, severe, mild and so on.
I want to use R (or a better package - I'm open to ideas) to perform some hypothesis-neutral analysis to find clusters or expression associations, if they in fact exist in the data.
My question is related to cleaning up the dataset so that R will won't throw an error when it is trying to undertake a hypothesis neutral clustering. I've tried entering "na" but that isn't helping.
So, over to The Community!
Many thanks
G
Grant,
Just to be sure, have you used "na" or "NA"? Because R is case sensitive. If that was your problem, just use NA (without quotation).
Hi Grant,
As mentioned by Alexandre, you will need to use NA instead of na in R. After giving the NA to the missing value row, you can use the is.na function to find all the missing values and then remove them from your data. If your function have the option of na.rm, you can set that to TRUE such that the function will automatically handle the NA
If you just have missing values, you don't need to do anything. The read.table (or read.csv) function should automatically detect them as NA values. If you used "na" (lower cases) you could also try adding na.strings="na" to the function.
Be aware, that if you use na.omit, or also na.rm in any case, the whole row of data would be removed from your observation. This becomes an issue if you have multiple parameters with differing missing values.
If you used "na" you can easily change these values to NA with something like this toy example:
x
If IL-2 was missing because it was undetectable, imputing the overall mean would be a bad idea-- IL-2 undetectable means the quantity present was below the lower limit of detection for the lab, so the overall mean of the non-missing values would be much too high. The "missingness" in this case is highly informative about the quantity being measured. One imputation commonly used in this situation is to impute 1/2 the lower limit of detection-- the lab that ran the IL-2 measures should be able to give you an estimate of that limit.
The missingness is also highly likely to be linked to type of patient-- many healthy people with low levels of inflammation will have low or undetectable levels of IL-2, while severely ill people with the immune system highly activated will have extremely high IL-2. For simple group comparisons of average level, consider using a rank-based statistic, like the Wilcoxon rank sums test, to compare median IL-2 after imputing very low values below the limit of detection. Otherwise, the log transformation may be of some use in normalizing the data after imputation for parametric methods.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis