I am working with a data-set that is not normally distributed (when tested by normality test) but appears to be normally distributed (when observing the histograms).
I wish to compare performance of two methods against each other, where there is no gold standard. My problem is, that whether I am working with the normal data or the log transformed data, I am still getting a non-normal distribution of the differences (as seen in the attachments).
1. I am confused as to whether I should follow the normal data, since the histogram of differences in that one looks slightly better than the log transformed one.
2. I read a few threads where people say the distribution does not really matter for the plot but rather the LOA. Does this mean I can use the same bias line from the normal data plot? Also if I am to assume non-normal distribution for the LOA, how do I calculate these limits? Is there a specific method/suggestion for non-parametric data?
Thanks
Hi Blessings Hendrina Likoswe
This is how I interpret your results.
1. The distribution of errors, with and without log transformation, is not normal. The histograms are usually misleading as what appears to be normal may turn out to be non-normal if we change the number of bins used to plot the histogram. Similarly, the tests for normality are also misleading as they are usually significant if the dataset is large enough. The QQ-plots are a good way to check the normality.
2. I would argue not to log transform the data simply because then you have to live in the log world and the units of measurement become log-units. If they have intuitive interpretation in your case, go ahead and log transform otherwise not.
3. From the Altman plots, a few conclusions can be drawn. There is a systematic bias in your measurement method 1 compared to 2, or the way around as you don't have a gold standard. This is, however, not the end of the road. This systematic bias can be accounted for by regression analysis which is reported as "y(differences) = -0.027 x(means) + -0.35" in the non-transformed data report. This can be interpreted as the systematic difference between the two methods at a measured value (means stand in for the true score).
4. As the errors are not distributed normally around the systematic trend (or the expected value of the differences), the assumption that there is a constant LOA around the systematic difference is null and void. LOA is computed as bias + 1.96 (z-score from standard normal distribution) * standard deviations of differences. The violation of this assumption implies that in a practical situation, you may expect smaller random error at smaller measured values and larger random errors in larger measured values, once the systematic error in the instrument has been accounted for. You still need to interpret the size of the error based on the application of the measurement methods.
My conclusion is that the reliability of the methods seems to be less than desirable. The methods may still be applicable with a mechanism to account for systematic errors and the random error is expected to be large for larger measurements.
Article Statistical-Methods For Assessing Agreement Between 2 Method...
Usman Rashid I was also not in support of log transforming the values because the interpretation relies on the units (to determine relevance of the bias).
My aim is to just compare the two in terms of performance i.e to determine if there really is a need to use both, or just one. I am just confused as to whether or not I would report the presented LOA's in this context , since the standard of reporting for B & A. I will have to figure out how this can be reported. Thanks for the assistance.
Jos Feys it's true that normality tests (in general) are not supported by some statisticians, unfortunately, this package which I used, has the built in function set to assess normality with a test (shapiro) which I could not ignore. I will try to assess the normality with other methods to see if I get different results
- Badges
- Science topic
- Similar topics
- Correction