Hi!
I'm currently working on my dissertation - it's on the gender differences in moral reasoning behind moral dilemmas associated with cybercrime. Each moral dilemma has 3-4 open questions where each participant had to write an answer and their reasoning behind the answer. I have coded these answers via themes/insights and have calculated the percentages behind these themes.
It is worth noting, that each question allowed multiple answers, therefore leading to participants having two or more codes associated with their answer.
My leading professor asked me to analyze these percentages to seek any statistical differences. My only lead is a Chi-square test, however, it probably doesn't seem the right choice because:
1. Each question is not limited to a yes/no answer, there are answers like: I don't know, or answers providing alternative solutions, thus omitting the aforementioned moral dilemma.
2. Participants often answered with multiple answers, that is they could answer yes and provide a reason but later on in their response answer no and also provide a reason.
What I need to know if there is a way to analyze statistically this data, what is more, if it's even possible to somehow analyze the percentages I've calculated of this qualitative data?
Hello Piotr,
The answer depends on several points. First, I suppose you could identify an ersatz set of response options (e.g., A = gave relevant answer and reason; B = gave multiple relevant answers/reasons; C = gave alternative solution; D = was unable or unwilling to address the question). Then, if you wanted to compare individual questions, you could use McNemar's test (a chi-square statistic that focuses on the disagreement among matched cases), or a statistic like Cohen's kappa (which quantifies agreement beyond the chance level).
However, statistical power of such a comparison (envisioning sets of 4 x 4 tables or the like) won't be very good unless you have a lot of cases in your sample. It might just be easier to report percents of the relative categories by scenario or by individual question and not worry about tests of significance.
Good luck with your work.
Dear Piotr Rutkowski ,
My only addition to David Morse 's answer is to suggest that you use a rapid prototyping approach: start with the most relevant pair of questions and the most common answers to them. This can probably be turned into a Ch-Squared test for which you will need to interpret the meaning of the association or lack thereof.
Then you need to consider how to increase the complexity of this initial model in a way that adds value to your analysis. There will probably be a cut off point where adding more complexity is not worth the increase in explanatory power (this is known as the principle of parsimony).
You can use the list of codes and create a present-absent dummy variable for each of them. This would allow you to do a chi-sq or t-Test of proportions for each code.
I think you don't need to use statistical test and using the percentage is enough, but the method suggested by David L Morgan,can be useful.
I think this is a good example of why you need a mixed methods approach. Your challenge in this, I assume is your methodology was the quantitative approach, thus you had to quantities qualitative data. To me, you are now seeking to statistically manipulate the data which could be meaningfully analysed using the qualitative techniques.
The answer to your question is based on the methodology the dissertation was founded on . At first glance it seems that a thematic analysis would lead to a more robust analysis of your questionnaire . If you were seeking a mixed method approach then the questions with scales that can be assigned continuous variables could be quantitatively analyzed and the open ended questions thematically analyzed to see if the outcomes agree . However as an examiner I would query the literature that informs the approach you proposed and examine the face validity of an attempt to ,quantifying semi structured questionnaires .
Nkululeko Fuyane and Michael Felix Noel The question of whether "quantizing" qualitative data counts as mixed methods has seen quite a bit of debate. For better or worse, all the major qualitative data programs now call their ability to export codes as spreadsheets a form mixed methods, so that creates still more confusion.
Better u can do average score or percentage by having the codes. Even though you can use those responses for recommendations of your dissertation.
There might be problems with the instrumentation in designing the questionnaire in line with your chosen variables frame of the study. And run a pilot study to ensure the content validity of your questionnaire and then you key in the responses by using excel and run smart PLS instead of SPSS.
First, encode the answers using the "Multiple Answers" method.
And then use either the dichotomy method or the categorical method
Followup - I used the Chi-Square Test for frequency of a given answer/code and tested the gender differences. I also have analysed my data qualitatively and exported codes. I am reasoning that this mixed methods approach is important, given the type of data I received in my Exploratory Research - I had no hypotheses - the aim was to show the status of gender differences in moral reasoning among Polish adolescents in moral dillemas related to cybercrime. Given this fact, could any confirm if my approach was the correct one?
Piotr Rutkowski You can't do without hypotheses, because you work with sample data, the result can be interpreted only with a certain probability
- Badges
- Science topic
- Similar topics
- Computer Science
- Computer Communications (Networks)
- Internet