I am doing a project on a therapeutic group, utilizing non-parametric ordinal data to account for changes in emotional distress. Unfortunately attendance has been somewhat inconsistent throughout the project, so much so that I am only left with a viable sample of n = 2. This is obviously too small for a sign test, Wilcoxon Sign Test, and the Mann-Whitney U Test. I am having trouble determining what statistical operations to compare the pretest to the post test. Any thoughts or feedback would be extremely helpful! Thanks in advance!
Hi, Jonathan,
Tests based on classical test theory are available for designs involving a single case measured at least twice. These methods have been used, for example, to compare a student's performance on standardized test sub-scales, to support efficacy claims in Patent applications, to evaluate human and animal inter-trial learning, and in clinical studies evaluating the efficacy of therapeutic interventions for individual participants (Dr. Mueser's paper is an example of evaluating therapeutic effectiveness).
Here are some papers:
Yarnold, P.R. (1982). On comparing interscale difference scores within a profile. Educational and Psychological Measurement, 42, 1037-1045.
Yarnold, P.R. (1988). Classical test theory methods for repeated-measures N=1 research designs. Educational and Psychological Measurement, 48, 913-919.
Mueser, K.T., Yarnold, P.R., & Foy, D.W. (1991). Statistical analysis for single-case designs: Evaluating outcomes of imaginal exposure treatment of chronic PTSD. Behavior Modification, 15, 134-155.
Yarnold, P.R. Statistical analysis for single-case designs. In: F.B. Bryant, L. Heath, E. Posavac, J. Edwards, E. Henderson, Y. Suarez-Balcazar, and S. Tindale (Eds.), Social Psychological Applications to Social Issues, Volume 2: Methodological Issues in Applied Social Research. New York, NY: Plenum, 1992, 177-197.
Yes, as David said, and I think you might have another problem as well. You said "...attendance has been somewhat inconsistent...." So you may have a bias problem with your sample. That is, those who attended best may have provided substantially different responses than those who didn't attend consistently would have provided.
Perhaps you could consider this to be a preliminary study, not quite a pilot study, to help you prepare to try again. You will need conditions to obtain a much bigger, and better sample. That still might then become your pilot study to help determine the sample size you will finally need.
No statistical test is usable. You are left with describing with descriptive statistics. But keep James R Knaub '2 comment in mind! Those who drop out are probably worse than those who completed. You should try anew. Let's say you get 10 respondents, you might call it a pilot study. You should also try to interview your drop-outs and/or do a intention-to-treat analysis. Example: if you started with 10 respondents, finished with 2 and one person was described as "success". Then you have only 10% success (1 out of 10), not 50% (1 out of 2).
Best
JK
n = 2 is not a sample.
n = 2 is not statistical.
n = 2 is nothing.
It is not usually the case that one asks participants one question "how emotionally distressed are you?" It is more typical to ask a variety of questions and the answers combined to gauge emotional distress. Sometimes this equation is known from the literature, other times it is developed through a method like principal component analysis.
The appropriate sample size is determined by the type of analysis and the strength of the signal to noise ratio. It also depends on the type of sampling, and other factors.
Read the methods sections from ten papers that have published work with a similar experimental design. In each case record their sample size. That is one strategy to estimate a minimum sample size when not enough is known about the system you need to study. If more is known, then something like GPower might help.
Hi, Jonathan,
Tests based on classical test theory are available for designs involving a single case measured at least twice. These methods have been used, for example, to compare a student's performance on standardized test sub-scales, to support efficacy claims in Patent applications, to evaluate human and animal inter-trial learning, and in clinical studies evaluating the efficacy of therapeutic interventions for individual participants (Dr. Mueser's paper is an example of evaluating therapeutic effectiveness).
Here are some papers:
Yarnold, P.R. (1982). On comparing interscale difference scores within a profile. Educational and Psychological Measurement, 42, 1037-1045.
Yarnold, P.R. (1988). Classical test theory methods for repeated-measures N=1 research designs. Educational and Psychological Measurement, 48, 913-919.
Mueser, K.T., Yarnold, P.R., & Foy, D.W. (1991). Statistical analysis for single-case designs: Evaluating outcomes of imaginal exposure treatment of chronic PTSD. Behavior Modification, 15, 134-155.
Yarnold, P.R. Statistical analysis for single-case designs. In: F.B. Bryant, L. Heath, E. Posavac, J. Edwards, E. Henderson, Y. Suarez-Balcazar, and S. Tindale (Eds.), Social Psychological Applications to Social Issues, Volume 2: Methodological Issues in Applied Social Research. New York, NY: Plenum, 1992, 177-197.
Two comments about the usefulness of your results (maybe these are the "not much" David Eugene Booth describes).
1. The procedure you used to end up with n = 2 clearly has problems. Do you have data on when the different people left the sample? For example, if you started with 100 and 2 left after session 1 on happiness, and 96 left after session 2 on sadness, this gives you information about people leaving the sample and may be helpful when conduct a new study.
2. Two might be enough to conclude something. Consider a couple of examples. If your hypothesis is all ravens are black, finding two white ravens would make me question this hypothesis (and the accompanying measurement hypothesis) more than just one (so unlike Gioacchino de Candia n = 2 is a sample, n = 2 is statistical, n = 2 is something). Second, suppose are looking at emotion distress of individuals in companies, and you expect variation among companies. You end up with only two companies (but say 1000 people in each). At one level, your n is two and you wouldn't want to make statistical generalization too much across companies, but you can look to see if the patterns of results are similar in the companies. This is essentially what happens in some psychophysics studies where a couple of subjects respond to thousands of trials.
These comments aside, for many purposes and likely the purpose of your study, there will not be much you can do with n=2 when the original design may have been for n=100. If you are required to write this up as is (without collecting more data), you might change to focus to subject retention.
Where n = 2 is problematic, but with small samples it is better to use Mann-Whitney U test
Daniel Wright in your "dissertation" you contradict yourself.
Needless to comment again.
n = 2 is not a sample and cannot produce statistics.
That's all folks!
Statistics have been developed in large part to summarize large amounts of data. When studies examine small data sets, descriptive analysis may be all that is possible. Small data sets can be used to develop theories, but larger collections of information are required for testing theories.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size