Hi all, I wanted to see whether extrinsic or intrinsic motivation affects the frequency of cosplaying. The questionnaire for motivation has 4 subscales, in which I mark them as E1 (social), E2 (social avoidance), I1 (intellectual), I2 (competency mastery). The frequency was measured as whether participants have cosplayed (meaning dressing up as a character at any event) once, twice, or more than thrice. Logically, intrinsic and extrinsic motivation influence each other, but I thought that it would be interesting to see whether there are any significant differences between each sub for this particular sample. In other words, it's like finding out, for example, which one contributes more to the behaviour of cosplaying; is it because of increasing skill mastery or a sense of belonging?
IV: types of motivation with 2 levels (intrinsic, extrinsic). possibly it's counted as 4 levels if I regard each subconstruct (social, social avoidance, intellectual, competency master) as 1 level.
DV: frequency of cosplay
My current hypotheses are kept simple for now:
H1: There is a relationship between extrinsic motivation and the frequency of cosplaying.
H2: There is a relationship between intrinsic motivation and the frequency of cosplaying.
I'm getting confused after all the research for a suitable test. For now, I'm clear that I have an ordinal IV and DV. However, I'm unsure where should I begin looking if I were to find the differences in effect or predicting which type (E1, E2, I1 or I2) of motivation subconstruct would contribute to the frequency of behaviour. I'm looking into ordinal regression, linear regression, ANOVA ... but none of them seem to be suitable. Originally, I thought of doing Pearson correlation since that seems to be an obvious choice, but I'd like to explore more about the 4 subconstructs. Unless I'm not looking at my whole picture correctly, I would really appreciate input and assistance. I'm more than happy to give more details about the research to help in this inquiry. Thank you!
As you mention, ordinal logistic regression should work in your case (ANOVAs and linear regression are out the windows as both your IV and DV are ordinal), if you want "the full picture" you could maybe (depending on your sample size and the distributions of your variables) re-code your ordinals into numeric and do a structural equation model
The two IVs have four sub levels each hence in effect it's 8 IVs. So the interaction with the DV will be at 8 different levels. Hence the research design will be of 4x4x1 structure.
Therefore you'll 16 sub-groups of IVs which can be analysed using 4 Way ANNOVA after suitable conversion of ordinality into specific "instances".
Thank you for everyone's input.
Marius Ole Johansen My sample size is fairly small (n = 124) so I doubt I can run SEM with the data. I'll still look it up and learn more about it. Glad to know that at least I can somewhat confirm ordinal regression to be the suitable test.
Kamal Karamchandani Actually, the 2 IVs have 2 sublevels, (E1, E2, I1, I2), which makes it 4 IVs in total. Additionally, as Marius has mentioned, ANOVA is not possible since both IV and DV are both ordinal, unless there are other things that I can look into.
If I summarize what I seem to understand, you have in your questionnaire:
- E1: social belonging (as a Likert scale or binary variable I assume?)
- E2: social avoidance (is this the opposite of E1?)
- I1: intellectual
- I2: competency mastery
- F: frequency of cosplaying (in three levels: once, twice, thrice or more)
- Intrinsinc motivation (IM): two levels (maybe low vs. high)
- Extrinsic motivation (EM): two levels (maybe low vs. high or yes vs. no)
I am not sure what you mean about considering E1, E2, I1 and I2 as 1 level. Do you mean to use them as binary (yes/no) variables?
It doesn't sound like you are comparing two groups if my understanding is correct. Rather, you are getting data from one sample of your target population, and trying to infere some relationship between your variables of interest in this population. And all your variables appear to be ordinal variables (E1, E2, I1, I2, F, IM, EM).
For this the first thing I would recommend is to check the Spearman rank-order correlation between IM and F, and between EM and F. It is also possible to test the statistical significance of the correlations, although this will only tell you "whether or not the correlation should be neglected".
Another possible way to analyze the results could be to test the specific effect of each variable separately by splitting your data based on the different levels of your variables. For example, you could split your data by levels of IM, then present the results for F as a table to see whether IM is having an effect on F. If the table is 2x2 you could analyze it with Fisher's exact test (to test for independence between the variables), or with Odds Ratios (to "quantify" the effect of one variable on the other). For bigger tables, the Chi2 test could do.
You could also split your data into two groups, either by splitting results based on the median of one variable, or by comparing one level to the rest. It's easy to do with binary variables notably, but for example, you could compare all other variables after splitting your sample into "F=once" vs. "F=twice or more". Then check for differences between the the groups you created with the Mann-Whitney U test. It's also possible to calculate Odds and confidence intervals for the odds from the U values (I explain how in my recently published article).
PS: as you and @Marius Ole Johansen pointed out, ordinal logistic regression with a maximum likelihood estimator is yet another possibility. But it might require stronger assumptions about the distribution of your variables of interest.
Yohann Opolka Thank you for your detailed answer. It took a while for me to digest it.
To clarify, the questionnaire has 4 subconstructs which I have marked as E1, E2, I1, I2. I'm not sure if I should consider each subconstruct as 1 level, and it would be helpful if you could advise if it's better to regard them as such. So maybe I can just treat each subscale as they are like:
E1, E2, I1, I2 ---> F
Instead of (assuming that EM = total of E1 and E2 combined. To be honest, I'm not sure if this is even doable):
EM ---> F
IM ---> F
The questionnaire is a 5-point Likert scale. Yes, I'm not comparing groups as I'm only using one sample to see the effect of motivation on the frequency of engagement in the hobby.
The total sum of the scores in the questionnaire is equivalent to whatever the subscale is measuring. So I assume I can run Spearman's Rank for each subscale separately instead? And to compare the "strength effect" by looking at the correlation efficient as it is?
Regarding ordinal regression, what do you mean by stronger assumptions about the distribution?
@Amirah Hata,
I think I see what you mean, but I would advise against using the sum of the scores for your measurement. If you consider that the measurements are ordinal, that implies that operations such as addition, substraction, product and division are not clearly defined for this type of data (because the only information you have is whether one thing is greater or lower than another).
That is, unless you are relying on a specific theory stating that the sum of these scores is a valid construct. (Are you?)
That's why I think it might be more suitable to use the Spearman correlation between all your "subconstructs", and see directly from the correlation score whether an effect is observed or not. If you want to calculate statistical significance, there is a way to calculate a p-value from the spearman correlation. You can find it on Wikipedia, and most statistical software calculate it automatically.
Regarding the ordinal regression, as with any type of regression, it relies on the assumption that either the data, or the "errors" between your data and the regression curve have a specific statistical distribution (e.g., normal distribution, logistic distribution, etc.). If I am not mistaken, ordinal regression for Likert scales assumes a logistic distribution of your data around the median. It also assumes that the scale is approximately continuous (e.g., that a score of 3.331 has a meaning, and that it is closer from 3 than it is from 4). This might or might not be a good approximation, depending on the psychometric properties of the scales you are using.
Nonparametric methods such as Spearman's rank-order correlation or the Mann-Whitney U statistic try to avoid these assumptions by using a rank transformation as the first step of the analysis (i.e., the Spearman correlation is basically a Pearson correlation after rank transformation).
if there are no obviously applicable model-structures with associated theory for significance-testing, then you could consider going back to the basic principles of statistics. Specifically, you consider the two steps separately: (a) finding a test-statistic that measures what you want measure; (b) finding a null-distribution for that test-statistic.
The question of finding a null-distribution for a test-statistic can always be solved by applying a randomization argument, and often a permutation approach would be suitable.
The question of applying a randomization approach when you are using some complicated model-fitting procedure is solved by applying the whole model-fitting procedure to each version of the randomized data-set. The model-fitting procedure gives some maximised or minimised goodness-of-fit value and you use this as the test-statistic.
In your application, the goodness-of-fit value might be based on the "ranked probability score", but you might want to construct a modified version of this tailored to your context.
You may wish to find an experienced statistician for face-to-face guidance.