I'm using McNemar tests to compare the proportions of dichotomous related samples. The problem is that not all test cases are represented in the second measure, is there a test that functions with missing data?
Running most analyses in SPSS so anything that runs in there would be preferred.
Tests with Two Independent Samples, Dichotomous Outcome
Here we consider the situation where there are two independent comparison groups and the outcome of interest is dichotomous (e.g., success/failure). The goal of the analysis is to compare proportions of successes between the two groups. The relevant sample data are the sample sizes in each comparison group (n1 and n2) and the sample proportions (equation image indicator) which are computed by taking the ratios of the numbers of successes to the sample sizes in each group, i.e.,
equation image indicatorand equation image indicator
There are several approaches that can be used to test hypotheses concerning two independent proportions. Here we present one approach - the chi-square test of independence is an alternative, equivalent, and perhaps more popular approach to the same analysis. Hypothesis testing with the chi-square test is addressed in the third module in this series: BS704_HypothesisTesting-ChiSquare.
In tests of hypothesis comparing proportions between two independent groups, one test is performed and results can be interpreted to apply to a risk difference, relative risk or odds ratio. As a reminder, the risk difference is computed by taking the difference in proportions between comparison groups, the risk ratio is computed by taking the ratio of proportions, and the odds ratio is computed by taking the ratio of the odds of success in the comparison groups. Because the null values for the risk difference, the risk ratio and the odds ratio are different, the hypotheses in tests of hypothesis look slightly different depending on which measure is used. When performing tests of hypothesis for the risk difference, relative risk or odds ratio, the convention is to label the exposed or treated group 1 and the unexposed or control group 2.
For example, suppose a study is designed to assess whether there is a significant difference in proportions in two independent comparison groups. The test of interest is as follows:
H0: p1 = p2 versus H1: p1 ≠ p2.
The following are the hypothesis for testing for a difference in proportions using the risk difference, the risk ratio and the odds ratio. First, the hypotheses above are equivalent to the following:
For the risk difference, H0: p1 - p2 = 0 versus H1: p1 - p2 ≠ 0 which are, by definition, equal to H0: RD = 0 versus H1: RD ≠ 0.
If an investigator wants to focus on the risk ratio, the equivalent hypotheses are H0: RR = 1 versus H1: RR ≠ 1.
If the investigator wants to focus on the odds ratio, the equivalent hypotheses are H0: OR = 1 versus H1: OR ≠ 1.
Suppose a test is performed to test H0: RD = 0 versus H1: RD ≠ 0 and the test rejects H0 at α=0.05. Based on this test we can conclude that there is significant evidence, α=0.05, of a difference in proportions, significant evidence that the risk difference is not zero, significant evidence that the risk ratio and odds ratio are not one. The risk difference is analogous to the difference in means when the outcome is continuous. Here the parameter of interest is the difference in proportions in the population, RD = p1-p2 and the null value for the risk difference is zero. In a test of hypothesis for the risk difference, the null hypothesis is always H0: RD = 0. This is equivalent to H0: RR = 1 and H0: OR = 1. In the research hypothesis, an investigator can hypothesize that the first proportion is larger than the second (H1: p 1 > p 2 , which is equivalent to H1: RD > 0, H1: RR > 1 and H1: OR > 1), that the first proportion is smaller than the second (H1: p 1 < p 2 , which is equivalent to H1: RD < 0, H1: RR < 1 and H1: OR < 1), or that the proportions are different (H1: p 1 ≠ p 2 , which is equivalent to H1: RD ≠ 0, H1: RR ≠ 1 and H1: OR ≠
1). The three different alternatives represent upper-, lower- and two-tailed tests, respectively.
The formula for the test of hypothesis for the difference in proportions is given below.
Test Statistics for Testing H0: p 1 = p
equation image indicator
Where equation image indicatoris the proportion of successes in sample 1, equation image indicatoris the proportion of successes in sample 2, and equation image indicatoris the proportion of successes in the pooled sample. equation image indicatoris computed by summing all of the successes and dividing by the total sample size, as follows:
equation image indicator
(this is similar to the pooled estimate of the standard deviation, Sp, used in two independent samples tests with a continuous outcome; just as Sp is in between s1 and s2, equation image indicatorwill be in between equation image indicatorand equation image indicator).
The formula above is appropriate for large samples, defined as at least 5 successes (np>5) and at least 5 failures (n(1-p>5)) in each of the two samples. If there are fewer than 5 successes or failures in either comparison group, then alternative procedures, called exact methods must be used to estimate the difference in population proportions.
Hello Rik,
I see some preprocessing options for your problem. The very first one is to remove samples/features with missing values. But only if the percentage of missing values is not to hight to the point of losing too many examples.
Continuing in this strategy, if you have a feature in which you have more than 50% of MV, you can remove this feature of your analysis in order to reduce the potential bias of the other solutions, or you can create an artificial boolean variable indicating if that feature is missing or not. This latter strategy is good to identify sensor issues - for example, a out of range value; or in cases that people are not comfortable to answer certain question due to its answer (e.g.: annual income - when it is too high)
The other option is to perform data imputation before your analysis. The best method depends on some datasets characteristics. Here are some methods that would be suitable for you: last value carried forward, knn imputation and another one based on Genetic Algorithm.
If you please, you can send me a message to further discussions.
Regards,
Fábio
I don't have SPSS on this machine, so cannot try anything right now. But I see that CROSSTABS (the procedure you would use to compute McNemar's Chi-square) is in the list of procedures that support pooling of estimates. So it may be possible to use multiple imputation.
https://www.ibm.com/support/knowledgecenter/en/SSLVMB_20.0.0/com.ibm.spss.statistics.help/mi_analysis.htm
By the way, for questions about how to do X with SPSS, I think you would be better off posting to the SPSSX-L mailing list. You can view the archive and join the list via Nabble:
http://spssx-discussion.1045642.n5.nabble.com/
HTH.
Tests with Two Independent Samples, Dichotomous Outcome
Here we consider the situation where there are two independent comparison groups and the outcome of interest is dichotomous (e.g., success/failure). The goal of the analysis is to compare proportions of successes between the two groups. The relevant sample data are the sample sizes in each comparison group (n1 and n2) and the sample proportions (equation image indicator) which are computed by taking the ratios of the numbers of successes to the sample sizes in each group, i.e.,
equation image indicatorand equation image indicator
There are several approaches that can be used to test hypotheses concerning two independent proportions. Here we present one approach - the chi-square test of independence is an alternative, equivalent, and perhaps more popular approach to the same analysis. Hypothesis testing with the chi-square test is addressed in the third module in this series: BS704_HypothesisTesting-ChiSquare.
In tests of hypothesis comparing proportions between two independent groups, one test is performed and results can be interpreted to apply to a risk difference, relative risk or odds ratio. As a reminder, the risk difference is computed by taking the difference in proportions between comparison groups, the risk ratio is computed by taking the ratio of proportions, and the odds ratio is computed by taking the ratio of the odds of success in the comparison groups. Because the null values for the risk difference, the risk ratio and the odds ratio are different, the hypotheses in tests of hypothesis look slightly different depending on which measure is used. When performing tests of hypothesis for the risk difference, relative risk or odds ratio, the convention is to label the exposed or treated group 1 and the unexposed or control group 2.
For example, suppose a study is designed to assess whether there is a significant difference in proportions in two independent comparison groups. The test of interest is as follows:
H0: p1 = p2 versus H1: p1 ≠ p2.
The following are the hypothesis for testing for a difference in proportions using the risk difference, the risk ratio and the odds ratio. First, the hypotheses above are equivalent to the following:
For the risk difference, H0: p1 - p2 = 0 versus H1: p1 - p2 ≠ 0 which are, by definition, equal to H0: RD = 0 versus H1: RD ≠ 0.
If an investigator wants to focus on the risk ratio, the equivalent hypotheses are H0: RR = 1 versus H1: RR ≠ 1.
If the investigator wants to focus on the odds ratio, the equivalent hypotheses are H0: OR = 1 versus H1: OR ≠ 1.
Suppose a test is performed to test H0: RD = 0 versus H1: RD ≠ 0 and the test rejects H0 at α=0.05. Based on this test we can conclude that there is significant evidence, α=0.05, of a difference in proportions, significant evidence that the risk difference is not zero, significant evidence that the risk ratio and odds ratio are not one. The risk difference is analogous to the difference in means when the outcome is continuous. Here the parameter of interest is the difference in proportions in the population, RD = p1-p2 and the null value for the risk difference is zero. In a test of hypothesis for the risk difference, the null hypothesis is always H0: RD = 0. This is equivalent to H0: RR = 1 and H0: OR = 1. In the research hypothesis, an investigator can hypothesize that the first proportion is larger than the second (H1: p 1 > p 2 , which is equivalent to H1: RD > 0, H1: RR > 1 and H1: OR > 1), that the first proportion is smaller than the second (H1: p 1 < p 2 , which is equivalent to H1: RD < 0, H1: RR < 1 and H1: OR < 1), or that the proportions are different (H1: p 1 ≠ p 2 , which is equivalent to H1: RD ≠ 0, H1: RR ≠ 1 and H1: OR ≠
1). The three different alternatives represent upper-, lower- and two-tailed tests, respectively.
The formula for the test of hypothesis for the difference in proportions is given below.
Test Statistics for Testing H0: p 1 = p
equation image indicator
Where equation image indicatoris the proportion of successes in sample 1, equation image indicatoris the proportion of successes in sample 2, and equation image indicatoris the proportion of successes in the pooled sample. equation image indicatoris computed by summing all of the successes and dividing by the total sample size, as follows:
equation image indicator
(this is similar to the pooled estimate of the standard deviation, Sp, used in two independent samples tests with a continuous outcome; just as Sp is in between s1 and s2, equation image indicatorwill be in between equation image indicatorand equation image indicator).
The formula above is appropriate for large samples, defined as at least 5 successes (np>5) and at least 5 failures (n(1-p>5)) in each of the two samples. If there are fewer than 5 successes or failures in either comparison group, then alternative procedures, called exact methods must be used to estimate the difference in population proportions.
Oh gosh, we have pishing here as well.
She extract her answer from: http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_HypothesisTest-Means-Proportions/BS704_HypothesisTest-Means-Proportions8.html
Probably the others are faking account.
Adriana's answer is regarding independent samples, though. What do you do when comparing two related dichotomous variables? I.e. data collected of same participants regarding the same information, a few years apart? What if, over those years, some of the participants have slipped away, you don't have an equal sample size anymore, but would be best to run the McNemar's test?
Szilvi Watson, you are correct in pointing out that tests for two independent samples are not appropriate. See my previous post in this thread--it discussed the possibility of using McNemar's test in conjunction with multiple imputation.
Bruce Weaver, thank you for your swift reply! I did end up running McNemar's, and with my extremely limited statistical knowledge assumed/noticed that SPSS excluded non-paired values and ran the test smoothly. I am most thankful. Hopefully, I will be able to report the results in an appropriate way :)
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS