This is a pre-intervention/post-intervention survey. For each question (total of 21) residents could score a max of 21 points (each question was 0-no, 1-yes). I want to calculate statistical significance for each question from pre-intervention scores and post-intervention scores. Let me know if you need more info.
McNemar's test is common for a single pre-post binary situation. You could do these for all 21 and then adjust for multiple comparisons, but the power would be low. Are the responses to these 21 associated? What is your actual interest? Not sure if doing significance tests on 21 items is useful here.
I would say a Paired Sample T-Test for each question. So you will have 21 tests. I am curious why to need to run for each question. Are the items for a single scale or multiple scales? Usually, testing pre- and post-test differences for a scale is more meaningful for a single item.
I'm not sure its useful either. As a whole the pre-test/post-test was significant using Wilcoxon-Rank. I was asked to make a clustered bar chart of each question (cluster 1=pre; cluster 2=post) with significance values for each question. I'm not even sure this is possible... For example, Question 1: did they introduce themselves to patient. Answer yes (1 point) or no (0 points). I added the scores for each resident (21 of them); so PreQ1=20 and PostQ1=21. (I realize these are super close, but there are others that have a larger difference.) Does that make sense? Daniel Wright Hua Lin
This is one of those situations in which it's important to figure out what is it you want to know, and how you want to think about the data.
In the original post, you suggested that each participant gets a score out of 21 in both pre- and post- test. In that case, it makes sense to add up the scores for each participant for both pre- and post- and analyze these with a paired t-test, if the data fits the assumptions of that test, or a wilcoxon signed-rank test, or even a paired sign test, depending on your hypothesis.
You could also use McNemar on individual questions, if that's what you want to know.
A more complex --- but potentially more informative approach --- would be to use logistic regression which has the potential to tease out the effects of both time and question.
You can do it using McNemar's, but whether it makes sense is another question. Whoever asked you to do it you should ask them what they mean by "significance."
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size