There are 31 items that were asked in this survey and were rated on a Likert scale. The same survey was used in the pre- and post test.
With 31 variables that need to be compared at two points in time, you would need to do a lot of non-parametric tests. As I suggested earlier, you should consider forming scales, which would be both more reliable and more compact than tests on individual variables.
Can you give us some more information?
- Are you interested in comparing the results item-by-item or are you interested in comparing the overall result on the survey?
- Did each question feature the same Likert-type scale or did different questions have different scales?
- Was each survey item of roughly the same difficulty for the respondent or do you need to adjust for the difficulty in answering?
In the simplest case where the items were of similar difficulty and used the same scale and you are only interested in compare pre-test total to post-test total, the usage of a paired t-test may be sufficient, but you should first check that you meed the distributional assumptions. I would suggest using a non-parametric, paired test for location such as the Wilcoxon Signed-Rank. It has fewer distributional assumptions than the paired-t and may be more applicable since your data is most likely not normal. It is easily accessible to non-statisticians and can be carried out in a wide variety of statistical packages. (It isn't even too bad to do by hand if you have too!)
If you're not willing to treat the individual Likert items as continous (this is a debated issue), then you could do item-by-item pre-post comparisons by running cross-tabs of each variable pre v. post. If you've created a summative score so that you have a continuous variable summarizing all the 31 pre and 31 post questions, then t-test might be fine. The biggest objection to summative scales/scores is that they gloss over any latent measures/traits (i.e., the factor structure) that was probably intended by the multi-item scale.
If this is a multi-item scale that others have used, check the literature on it and see what's been done. There are probably papers on the factor structure of the scale.
If these 31 items were created by you, have you done any psychometric evaluation of them. After item-by-item comparisons, I would explore the factor structure at each time point separately and see what you have.
Obviously, this is easier to do if you have some research questions you want to test, rather than making it a data modelling task (unless that's your thing). Depending on your purpose and outlet, single item differences might be fine (rather than factor analysis of pca). But if you do that be sure you control for multiple testing with post-hoc tests.
Good luck!
YES WE HAVE TO ADD QUESTIONS WHICH ARE RELATED WITH RE-TESTING AND THEN POST-TESTING AND THEN FIND TOTAL SCORE AND RUN THE ULTIMATE TEST.
You could try multiple correspondence analysis of pre situation to observe how they segment (or cluster), then try the post situation (same variables to see if they segment in same way ( or cluster) - this will enable comparisons for item by item - you can get more accuracy as coordinate values can be obtained for each item, giving more accuracy than just the same or different segments. I think far more informative than chi-square.
if you assume scale (can endorse by observing normally distributed) then can do pre/post by t-test?
One basic approach would be to create a set of pre- and post-test measures, and then use a paired-comparisons t-Test to analyze for change.
In order to do that, you would need to convert your 31 items into an identical set of scales for both time periods. The two most effective tools for creating those kinds of scales are Cronbach's alpha and Exploratory Factor Analysis. Because there have been so many questions about those techniques, I have started a thread here that provides resources on both of them:
https://www.researchgate.net/post/How_can_one_help_people_create_simple_scales_from_Likert-scored_items
To: Maha Pervaz Iqbal
A better choice is the nonparametric Wilcoxon signed-rank test.
Thank you all. I am reflecting on all the comments and will work on my analysis.
This series of answer just confirm some areas I was confused. Thanks. But Saad, I guess the nonparametric test can only be used when the assumption of normality is not met.
Michael, I have a similar statistical project, which test did you choose?
Non-parametric tests are the best methods to use in situations like this.
Your question seems to contradict your second sentence?
If the variables are on a Likert scale they can be considered scale variables (if normally distributed). If the pre and post tests are matched the paired t-test can be used, if not matched an independent t-test can be used.
There are a lot of ifs here that need addressing.
1. You say categorical variables - what is the nature of these categorical variables (nominal or a rating)
2. If nominal then should ignore the opening paragraph here and seek a non-parametric approach
3. If a rating what is the scale - is it Likert (5/7 point)? - If the latter then opening paragraph is a possible route to analysis
More information needed?
It is clear that you ordinal data (likert scale). I agree with Olusegun and McClelland. As McClelland indicated, you have to watch whether or not the pre- and post-samples are dependent (matched). If matched, can use the Wilcoxon signed-rank or the paired t-test. If not matched, can use the Mann Whitney (equivalent the Wilcoxon rank sum) as Olusegun suggested.
With 31 variables that need to be compared at two points in time, you would need to do a lot of non-parametric tests. As I suggested earlier, you should consider forming scales, which would be both more reliable and more compact than tests on individual variables.
Can someone give me some insight as to what I should do for a similar problem but I have different instructors providing the lessons? I'd like to see how scores relate to which instructor the students had
This may be a different question but are there any suggestions as to how continuous variables can be built into a simple pretest postest measure replacing the problematic likert scales and allowing paired sample testing
Hello All. I have read the thread, thank you for the useful suggestions.
I am also doing a pre and post intervention survey, that is using likert scale responses to create a final continuous index number. I did not create the scale, and factor analysis has already been done to confirm validity.
I have several questions.
1. I am confused as to whether my samples are paired or un-paired. I used the same exact survey, however I used random sampling within the same neighborhood for both data collection dates. Therefore, while my sample is meant to reflect the same population, the respondents cannot be compared 1-1. What would this be?
2. Following that, I know a t-test could be used to compare final results. How else could I compare the survey responses? I am assuming that multiple regression would be appropriate to see how each variable impacts the outcome, however how could I go about comparing the TWO surveys?
Thank you in advance for comments and suggestions.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics