Hi there,
I've collected data through a survey in which we asked participants to rate behaviours on a couple of dimensions (e.g. effectiveness of engaging in the behaviour). Specifically, each participant rated 3 of 20 behaviours which we randomly allocated so that each of the 20 behaviours has 50 (different) participant ratings. The data set I received back is a lot richer and more complex than I had anticipated.
The challenge I believe I have is that there are participant clusters (the same participant rating three different behaviours). Additionally, I wonder whether I should also include behaviour fixed effects, for example when looking at the effectiveness aggregated across all of the 20 behaviours?
For reference, two of the questions I am trying to answer are the following:
1. I would like to use t-tests to compare groups of people (e.g. participants identifying as men vs. those identifying as women) against one another and see if they rate the behaviours overall (in aggregate) differently (e.g. more or less effective).
2. I would like to run a regression to understand how people judge the behaviours (i.e. which factors matter).
Hoping the above makes sense and someone might be able to point me in the right direction. If there is anything I can clarify further I'd be more than happy to do so.
Many thanks!
Based on the information provided, it sounds like you have a clustered data set with multiple observations per participant, and you would like to use t-tests and regression analysis to examine differences in ratings of behaviors between different groups of participants and understand which factors influence how participants rate the behaviors.
To account for the clustering of the data, you may want to consider using mixed-effects models or multilevel models, which take into account the fact that there are multiple observations per participant. This will help control for the non-independence of the observations and give you more accurate standard errors for your estimates.
In terms of comparing ratings of behaviors between different groups of participants, you can use t-tests to compare the means of the ratings for the different groups, but you will need to make sure to account for the clustering in the data when calculating the standard errors for the means.
For the regression analysis, you can include behavior fixed effects in your model to control for any systematic differences in ratings across the different behaviors. Additionally, you can include other predictor variables, such as participant demographics or other characteristics, to understand which factors influence how participants rate the behaviors.
It's also worth considering exploring some visualization tools like heat map or box plots, which will help to understand the distribution of the variables and the patterns in the data.
It's also important to note that the sample size of 50 observations per behavior may be small and might affect the power of the statistical analysis.
- Badges
- Science topic
- Similar topics
- Statistics
- Survey