Sometimes it will be obvious, but not always in my experience.
A recent study I was reading implied it was 400 (and it was on this basis the inferential tests were undertaken). However, the study was exploring how each of 400 student assessments was marked according to the gender of the student and the gender of the marker (to see if there is a 2x2 gender bias in marking of assessments). There were 8 markers (4 female and 4 male).
Couldn't this also be interpreted as the N actually being 8? I'd have thought it was more 'honest' to interpret it that way, as the 400 observations would not be independent (50 were carried out by each marker).
I've struggled with this question a few times and it seems like what is the unit of analysis can be ambiguous.
Really interesting question, Andrew! Maybe sample size is complicated from how you look at it. Is the hypothesis about the students or markers? If it's about predicting the students' marks, there are 400 participants. If the marks are subjective (involve coding), maybe marker matters for strictness vs leniency or bias toward a sub-group (like gender). Your analysis might be a 2 (student gender) by 8 (coder) ANOVA to predict mark. If your hypothesis is about the markers, you have 8 participants. But I don't see how the study you're describing could test for gender bias? You'd need a true score for comparison or multiple coders for the same student's work. Even multiple coders wouldn't tell you about gender bias because maybe there is a gender difference in what's marked among the students. If I were to test markers for gender bias, I'd probably give all of them the same assessments and for each assessment I'd randomly assign a student gender (e.g., some markers grade "Andrew's" test and some grade "Abigail's," even though they're actually the same test). Nice discussion! ~ Katie
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation