For such vast data it would be a good thing to combine chi-square tests for independence with chi-square goodness-of-fit tests. The former is for test of association between variables and the later is for testing goodness-of-fit of an empirical distribution to a given hypothesized one.

Ette, thank you very much for your response! So, if the test of association will tell me if there is an association between gender and having counselling, for example, would the goodness of fit test not tell me the same thing?

In the goodness of fit test, is the hypothesized distribution dictated by the actual distribution in the population? But the test of association already takes that into account (as in order to run it, I shall create variables of frequencies - e.g. number of males who went for counselling vs no of males who did not; no of females who went for counselling vs no of females who did not, etc). Am I missing something?

Thanks a lot!

I'm sure I don't count as a clever person, but...

It sounds like what you would really like is to build a model with count of Sessions as your dependent variable and Gender, Level, Instrument, and Nationality as independent variables. This can be accomplished with e.g. negative binominal regression. This is a something akin to Poisson regression, a type of generalized linear model. And it's not as scary as it sounds.

Edit:

For the relationship between the counts of counseling sessions and the individual factors, a Kruskal-Wallis test, with Dunn test (1964) as a post-hoc, would likely make more sense.

Don't use median split. It just throws away information.

Using the goodness-of-fit test the way you suggest is likely to give you some interesting results. It tells you something very different than the test of association.

Salvatore, I disagree - I think you are clever indeed! I had not thought about the Kruskal-Wallos test at all, so thanks a lot!

I'll look into the negative binominal regression, then!

As for the goodness of fit test, I think I am missing something. So, if the test of association will tell me if there is an association between gender and having counselling, for example, would the goodness of fit test not tell me the same thing?

In the goodness of fit test, my hypothesized distribution is the actual distribution in the population (on which I already have the data). But the test of association already takes that into account (as in order to run it, I shall create variables of frequencies - e.g. number of males who went for counselling vs no of males who did not; no of females who went for counselling vs no of females who did not, etc). Am I missing something?

Many thanks!

Hi, Raluca.

I think you are right on the goodness of fit test. I didn't realize you were going to also count the number of people who did not have counseling. The two tests --- chi-square of association and g-o-f --- should essentially get at the same thing, but the hypothesis tested is somewhat different. The reasons to choose one or the other might just be practical. But also, for the g-o-f, you are essentially thinking of the sample size of the school population as very large relative to those who seek counseling. If this is true, I like the g-o-f idea. But if the counts of those seeking counseling and the population of the school are, say, within a couple orders of magnitude, I might opt for the test of association. That's not an established rule; I just made it up.

Plot of data fed in to a goodness of fit test, hypothetical data:

http://rcompanion.org/handbook/images/image301.png

Salvatore,

Thanks a lot for your reply! When you say "the two tests should essentially get at the same thing", I assume that you are referring to the goodness of fit and the test of association?

Would you suggest that I only conduct a Kruskall-Wallis test without the goodness of fit or test of association? Perhaps there would otherwise be too much overlap in the findings?

Many thanks!

I imagine you want to address both kinds of questions: 1) in which your response variable is dichotomous: counseling/no-counseling; 2) in which the response is the count of sessions (and would include only those who went to counseling). Two different questions; both potentially interesting. It's possible that the results would come out similar for both, but there's no reason not to conduct both types of analysis and then decide what's meaningful.

Whether you go with the larger model or not *, you also want to check for associations among the independent variables. One reason is that correlated independent variables can play havoc with the model. But even for the individual tests, correlation among the independent variables will affect the interpretation. In an extreme case, if all the UK people were male and all the EU people were female, you'd be absolutely unable to disentangle the effects of these two factors.

Finally, let me mention. I imagine you have a pretty large sample size. If so, don't read too much in to the p-values. With a large sample size, you are likely to get small p-values even for small effects. You want to bring some measures of effect size to bear. This could be differences in proportion, say, or differences in count.

But you also may want to look at effects size statistics --- phi and Cramer's V for nominal/nominal; and for nominal/ordinal case maybe Cliff's delta, Vargha and Delaney's A, Freeman's theta, epsilon-squared.

The upshot here is that you may find a statistically significant effect, say the proportion seeking counseling by sex. But if the difference is 1.0% of males and 1.1% of females, this may not be something of very much interest. Reporting the proportions or a phi value will convey this. And ultimately, it's your judgement that determines if an effect is meaningful or not. No statistic can replace this.

_______________

The question on the two tests, association and gof: yes. I edited my answer, maybe after you read it.

______________

* As I mentioned in my first response, my inclination would be to construct a model with all the independent variables in the model together. Thinking about it, you would need to know all the cross-tabs for the demographic information. I mean by that, how many are male/UK/strings vs. how many are male/UK/wind, and so on. I suspect you might have this information for 2) (those who went to counseling), but may not for 1) (the entire school population).

Salvatore,

One thing is certain: you will go right into the Acknowledgements section of my thesis!

I immensely appreciate your altruistic guidance, which, without exaggeration, is proving to be key in my approach to this set of data!

You have given me so much food for thought - your messages so keep me awake until late and I thank you for that (they also speed up my understanding of these matters!)

I think I shall postpone the construction of a model with all independent variables for now, as I have to speed up the writing of this chapter and this set of data represents a very small part of my thesis. It seems much more relevant for my purposes to, as you rightly point out:

1) investigate associations between the independent variables (I assume for the data set of only those who did do counselling and, I assume, chi square test of association between a) gender and level; b) gender and nationality; c) gender and instrument; d) level and nationality; e) level and instrument; f) nationality and instrument)

2) Relationships between independent variables and counselling/no counselling as dependent variable)

3) Relationships between counselling counts and relevant independent variables.

I will get back to you on the issue of a larger model if/when needed!

Your help could not be more appreciated!

Thanks for the kind words. If you need to get a hold of me, sending a direct message is good idea as I don't get alerts when posts are updated.

Excellent - thank you so much, Salvatore!