Does a Box-Cox transformation make it easier or more problematic to compare the outcomes of multiple research projects?
(Here is a hypothetical situation, if that helps to answer the question.)
I have a research project where I have 20 independent variables and 1 dependent variable. I have 6 treatments, and therefore with a multiple comparison proceedure has 15 pair-wise tests. I am interested in all the univariate models. I am also interested in knowing about group differences, so discriminant analysis is useful, and there are some options for multiple regression as well.
I have 5 friends doing the same thing (6 total projects), but we can only "collaborate" through the literature. I don't get to see their raw data, and they cannot see mine.
None of the residuals are normally distributed, but a log transformation often corrects the problem.
1) we could all use a log transformation for all the data.
2) we could try to optimize and for one of the variables 4 of the 6 projects will use log, one will not use any transformation and another will use square root transformation. The trend continues where most of the projects use log, but one or two will find that another transformation of the data seems to work better.
3) we could use Box-Cox transformation. None of the projects have the same exponent, but all exponents are within the 95% confidence interval for at least one other project.
The sample size is whatever you need to make it in order to answer the question.
I now want to compare the results from the different projects. I am interested in knowing if the conclusions were the same, and if the magnitude of significant effects are similar. I am also interested in back-transforming some of the LSmeans so that I can directly compare values.
Well, that hypothetical situation is hypothetically bad science (or highlighning a crucially bad point about current scientific practoce)... good science is based on sharing data...
But to your problem: it is generally not helpful for making results comparable if the analysis (e.g. transformation) depends on features of the data at hand. Ideally, the analysis should be driven by theoretical considerations based on a thorough understanding of the data (and the processes behind) (but I admit that sometimes an exploratory look at the data is all we have, and at some point we just need to decide what to do).
Option 1 would work, 2 and 3 are bad ideas.
PS: I assume that the response variable is the same in all cases, measured with the same technique/assay under comparablr conditions (except for the experimental factors).
@Jochen, Your assumption is correct. All variables are measured using the same methods within the limits of "people" to duplicate the approach. The people might be on different continents, so climate may be different, but the cultivar of plant used will be the same, as will the species of insect (no known biotypes, but no one has really looked).
Thank you
I recently gave a graduate class some data and asked them to use Box-Cox transformation(s) to get approximately normal data. Students returned data with different exponents as they used different macros for BC transformations. In the end, I suggested to my students to forget about BC transformations and to use instead a normal quantile transformation (normal scores). This would be my suggestion to your question too, Timothy. Using such an approach may smoothen out some differences between the six projects that are due to some uncontrolled secondary factors of variation.
The Box-Cox transformation was originally intended to transform data that are normal to be closer to normality, or data that are heteroskedastic to be closer to homoskedasticity.
The Box-Cox transformation was not intended to make different studies comparable.
While comparison was not the intent of Box-Cox, comparison is what we do. How much of the difference between two outcomes was biology, how much experimental design or technique, and how much of the difference was an artifact of using a slightly different transformation coefficient?
If I run an experiment to study the relationship between A and B, then I would expect that each time I run the experiment I will get a slightly different Box-Cox coefficient. There will be a mean value for the coefficient, with some descriptor of the deviation about this mean. However, any two studies might end up with a very different coefficient for no other reason than the sample size is two.
If the sample size is 2, there is no point in estimating any parameters, especially in the Box-Cox framework.
The maximum likelihood estimator is designed to have asymptotic properties as the sample size approaches infinity.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size