Dear researchers and collaborators
I conducted a study with purpose of “evaluation of psychometric properties of X Scale…”. “X scale” is a scale that has been designed and it is still being used to assessment of depression in especial patients.
In this study, one of demographic variables is " history of depression disorder: Yes/No” and other variable is "History of Antidepressant use: Yes/No”.
Now, I want to know that can I determinate criterion (predictive validity) validity by ROC?
For instance, I assess sensitivity and specificity, accuracy and cutoff point for scores of X scale according to " history of depression disorder: Yes/No”.
The results showed:
Area Under the Curve: .478, p value: .453, CI95%: .42-.53
The score of positive cutoff point with the Sensitivity (52%) and the Specificity (51%) was 105.5.
According to the results, can it be concluded that predictive validity of X scale don’t fulfill?
I want to know your opinion in this subject and interpretation
Thanking you in anticipation
Hamid
ROC curves are used for this purpose, so the basic answer is "yes."
Sensitivity and specificity figures near 50% are certainly discouraging. Usually one wants to know the base rate of the condition before evaluating these statistics, but when they're near 50% it probably doesn't matter. You can say that the depression scale does not predict the variables you're using as criteria.
My only question, then, is about those criterion variables. It sounds like you have two separate ones (history of depression and history of antidepressant use). Did you run analyses for each separately, or combine them into one dichotomous variable? Either way would be all right, I suppose. But here's the issue: Neither one asks whether the person is currently depressed. Nor do they rule out the possibility of an undiagnosed depressive disorder. All you're asking about is whether the person has ever actually been diagnosed with depression.
You see the problem? What if I was depressed a couple of years ago and got over it, or was treated successfully? There is no reason why I should be especially depressed now. But you would list me in the depressive group. And although it might be less common, what if I am depressed now, but haven't sought assistance? I'd end up in the non-depressed group.
Bottom line: You have a fairly interesting finding, but I wouldn't make too much of it. What's needed is an actual evaluation of the participants' current depressive symptoms, one conducted separately from the self-report you're studying.
@Stephen Joy
The subject that i raised is an example to clarify my question.
you are right.
Sincerely yours
Hamid
I agree with Stephen Joy. You only need the two questions "yes" or "no" to establish that the person has had a history of depression. To know whether the person is now depressed, you need a different measurement instrument systematically asking about current symptomatology.
I agree with the responders to your question. It seems to me that you have reached a high level of handling issues of criterion validity that you know more about than I do.
I do appreciate your sending me your research and commend you on the high quality.
- Badges
- Science topic
- Similar topics
- Biological Science
- Physiology
- Physiological Processes
- Stress