I am working on the comparison of methods for biochemical analysis. I found that in some literature R² value from graph is used to decide the accuracy of the method. Is it the best way to compare biological methods? If someone has any idea regarding this issue please advise me.
Typical methods used for analytical comparison is Deming regression or Passing bablok . Passing Bablok regression - you can compare two analytical methods to determine bias (systematic error). Unlike linear regression, Passing Bablok regression is ideal for comparing clinical methods because it allows measurement error (imprecision) in both the X and Y variables, does not assume measurement error is normally distributed, and is robust against outliers. Deming regression assumes constant precision (constant SD) over the measurement range.
You should also do Altman and Bland plot (bias plot) to assess bias across the measurement range. Other factors to consider is what type of test you are evaluating? i.e. screening, diagnostic, monitoring. The clinical requirements for acceptability will be determined by all these factors.
In case of analytical method equivalancy, general acceptance criteria is simple comparitive results ie % RSD & Absolute difference in mean result of two method.Otherwise, Student t- test for comparing the results of two methods. This may helpful to evualuate biochemical method comparison.
Suggest you look at either PDA Technical Report 57 "Analytical Method Validation and Transfer for Biotechnology Products" or USP for guidance. Several approaches are acceptable but keep in mind you need sound scientific and statistical rationale/justification for the approach you take. A solid definition of what you mean by comparable is also required since you may want to show one of three possible scenarios i.e. (1) non-inferior, (2) equivalent or (3) supperior. Additionally you must take into account whether or not the method is qualitative (produces pass/fail results or result not more than X) or is quantitative.
In our method comparison studies we use the following criteria. (1) Sy/x for within-run imprecission, (2) Bias at 3 concentration levels (average, lowest and highest), and (3) TE (total error). We don't use Rsquare, this is a good measure for correlation of data, but not for linearity. I would suggest reading this paper: "Statistical methods for assessing agreement between two methods of clinical measurement." by Bland and Altman (1986)
Regression and correlation analysis are useful. Along with these calculations of sensitivity, specificity and kappa values might also be useful for comparing the methods.
Statisticians tell us that, in general, R^2 is not a good way to assess method agreement. A better measure is said to be Sxy.
Also, the choice of regression method depends on several factors. If error is mostly concentrated in the y-axis (typically the new method) then ordinary linear regression is appropriate. If error is about equally distributed between x and y then Demming regression is appropriate.
Both ordinary and Demming regression are very sensitive to outliers. One very bad point can ruin the comparison. Therefore, it is sometimes more appropriate to use more robust methods of regression. One such method that is popular for method comparision in the field of clinical chemistry is known as Passing-Bablok regression. It supposedly gives non-biased estimators of slope and intercept and is also robust against a small number of outliers.
Passing-Bablok regression also has the advantage of making relatively few assumptions about the distribution of errors in the variables, unlike Demming or ordinary regression.
As is almost always the case, whenever you gain an advantage in one area you typically lose something in another. Although I am not a statistician and can't say this authoritatively, I strongly suspect that if the underlying statistical assumptions of Demming or ordinary regression are met, those methods probably produce better fits to the data. However, if those assumptions are not met then I suspect that Passing-Bablok is probably more reliable.
1. Sy/x for within-run imprecission,
2. Bias at 3 concentration levels (average, lowest and highest)
3. TE (total error).
Additionally, Passing-Bablok regression is useful..
http://www.biyokimyadoktoru.com
In clinical laboratories in the USA, Sxy is the standard procedure for comparison of methods, 30 days of tests are required. A rule of thumb is that the value should be 0.96 or greater. Of course, one must take the sensitivity and specificity of the new method into account; a ROC curve that is not pronounced says that there are too many false positives and false negatives.
I agree with Alan and Kemal, but Ithink you need, in adition:
Detection capability (low amount of substance that can be measured as non-zero).
Quantification capability (less amount of substance that can be measured with an acceptable CV, generally less than 10%)
Linearity.
Dynamic range.
ROC curves for detection of the situation under study.
All these parameters, together with those already mentioned by the other partners can help you decide which method is best for you and your condition.
Have also a look in:
Wu, C. & Zhen Yu, J. 2018, "Evaluation of linear regression techniques for atmospheric applications: The importance of appropriate weighting", Atmospheric Measurement Techniques, vol. 11, no. 2, pp. 1233-1250.
- Badges
- Science topic
- Similar topics
- Microbiology
- Assays