Is it possible to do a dependent t-test if pre & post were anonymous? Are there any alternatives?
Hello António Calha ,
Unfortunately, if you cannot match cases for the two scores, and you don't have all of the relevant summary statistics (Mean & SD for each score, AND correlation of the matched scores), there is no way to run a dependent or paired t-test.
You can run an independent groups t-test, with the understanding that it will not be as sensitive to the differences, since person variation is now a part of the "error" term. In general, comparing independent and dependent t on matched data, dependent t is more sensitive (as long as the two scores have a correlation > 0).
This is a challenge in research when maintaining confidentiality of response is vital. There are tactics that are used (and which can pass review by IRB committees), such as: (a) having each respondent create their own pseudonym as an identifier (of course, they can and sometimes do forget what they've chosen across occasions); or (b) having a third party (independent of the researcher) maintain a master list of names and assign neutral IDs to cases while the study is in progress (thereafter the list of names is destroyed).
Good luck with your work.
Depending on context, you may be able to hand out some identifier to the participants. If participants then provide that identifier with their answers, that does not affect anonymity but you can still relate the two answer sets.
For example, in one case I know a set of envelopes containing arbitrary (unique) numbers was prepared. Participants drew a random envelope from the pile, so nobody other than the participant knew which number was assigned to which participant.
See also: https://www.researchgate.net/post/Which_statistical_test_should_be_used_on_2_levels_if_the_participants_are_anonymous
The original poster has probably moved on and forgotten about this question. But I'll add this for the benefit of others who find the discussion later.
If you don't know the correlation between the paired scores, but can estimate a range of plausible values for it, you can estimate a corresponding range of plausible paired t-test results. The attached text file has an example done using Stata. HTH.
Bruce Weaver , wouldn't it be better to simulate the "t"-distribution under H0 using the correlation coefficient as a random variable (with some resonable distribution reresenting the knowledge/uncertinty about its value) and determine the p-value from this distribution directly?
Hi Jochen Wilhelm. My initial response to your question is, I don't know! ;-)
Bruce Weaver - Cool, so I am now waiting for your intermediate response, right?
;)
- Badges
- Science topic
- Similar topics
- Statistics
- Survey