I am conducting a systematic review of a diagnostic test and some of the original research using latent class analysis as the "reference test". When using QUADAS2 as the tool for risk of bias assessment, I feel like there is something missing when the questions about the reference test come by. For example: Does the reference test correctly classify the condition of interest? The reference test was interpreted without the knowledge of the index test result? Is there a concern that the reference does not correspond to the review question? Was there appropriate timing between the reference and the index test? Were all subjects submitted to the same reference? Well I guess reasonable answers to these questions would be: unknown; no; depending on the question; depending on the other tests included, respectively. However, as I am not very familiar with the inner algorithms of latent class, perhaps some other opinions may give some light to these interpretations.
Hi Pedro,
This is a tricky question - as many can be with the QUADAS-2 from my experience due to the at times subjective decisions that need to be made when there are grey areas to interpret, or when the original studies do not provide sufficient information. My general advice would be to justify an approach and document it clearly and be consistent with it, as there is likely no right answer in this case. I will qualify my response with the fact I have never used LTA myself, but have read a number of studies using LTA and have read about them in considering them for past and future research.
Specific to whether the reference standard is likely to correctly classify the condition,
I don't know what your outcome is, or have the luxury of information about the development of the latent classes, but I think you need to consider both of these considerations. For example, I have done research on mental health screening tools, where some have questioned whether the structured interviews are always in fact accurate. In this case, I could see situations where a well conducted LTA would lead to a similar classification accuracy for my outcome/reference standard. In many cases diagnostic tools that are used are in essence LTAs that have been developed by creating clusters of symptoms. If the LTA uses similar symptoms, and creates meaningful and interpretable classes then I would be inclined to see this as accurate, especially if the class structure has been replicated elsewhere. The other thing to consider is that if the LTA uses reliable measures, then I would be more inclined to consider the LTA as accurate. In the case of mental illness, if well validated measures of mental health needs such as the Beck Depression Inventory, etc. were used then I would be more likely to consider the LTA accurate than if the researchers simply created questions and put them into an LTA and created a total data driven model that will inevitably be over-fit to the existing data. The other issue I would see is that LTA fits people into the best class (after the researcher chooses the best number of classes, which is not always a clear choice - I would suggest looking up literature on approaches to choose the number of classes. There is inevitably some error associated with that. The fewer classes there are, the greater potential there would seem to be for misclassification. For example, if you only have a healthy and a diseased class out of an LTA, grey zone cases might end up scattered between the two classes, potentially with a lot of error. The same thing applies to some extent however with diagnostic tools - for example if the symptom cut-off is 5 for a diagnosis, is a person with 4 symptoms really that different from a person with 5? They may in fact be more like a person with 5 symptoms than a person with 1 symptom, even though they would end up in the same class as the person with 1 symptom if a simple cut-off score is used.
The other questions seem more clear to me. As long as all participants completed the same assessments that go into the LTA, then you can say that all participants got the same reference standard. Similarly, the LTA would seem to be independent of the index test, as long as the index test isn't one of the variables included in creating the latent classes (in that case, then no it's not independent). The only other issue would be if the researcher went through individual cases in deciding whether to choose say 4 or 5 classes, and picked the one that led to the best performance of the diagnostic test, although this would seem highly unlikely. As long as the LTA creates classes that correspond to your outcome (e.g. different levels of severity of the disease) then this would seem to correspond to the review question. I would say that all of these questions depend less on assessing the LTA and more on assessing the measures that were administered and used to create the LTA. All that LTA does is take multiple measures and create one "variable" (or class in the LTA terminology) that summarizes all of this information.
Hopefully this is helpful - it's an interesting question, and I will be interested to see the views of others.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis