Your approach of comparing the demographics of the groups is not likely to be satisfactory.

First, with a rather large sample size, you are likely to find a significant difference in some demographic parameter. But this doesn't tell you if this difference is important in any difference in your dependent variable.

Second, statistically or logically speaking, failing to find a significant difference isn't really evidence that there is no difference, just that you were unable to find the difference. It could be, for example, that the test had a small sample size or low power.

A better approach is to build a single model that includes both treatments and the demographic information. This will tell you which of these is relatively more important in determining the value of your DV.

If that's too complicated, you might look at the association of each demographic variable, and treatment variable, with your dependent variable. Treat age or education as ordered if applicable. It is good to have an effect size statistic for these as well (for example, Spearman's rho for the correlation of age and DV).

Hey Salvatore S. Mangiafico thanks for your complete contribution, really appreciate it. It makes sense.

I like the approach you suggested in the last paragraph it's more aligned with what I am looking for, but I am having trouble understand the information. Can you further elaborate, please?

You are suggesting to do a Spearman's rho for the correlation of each demographic and the DV, is that it?

Mercedes Orús-Lacort thank you so much for your response. Chi-Square looks like a good place to start.

Regarding the t-test, seems an interesting approach, since I have ordinal data and therefore no normality, is there any non parametric test that I should use to do this comparison (that works only with nominal variables)?

To elaborate on my last paragraph, yes, I mean find the association between each independent variable (IV) and the dependent variable (DV).

This can be done as a first step of the analysis.

I will assume you are treating your DV as an ordinal variable. (You may be treating it as an interval variable).

Age is an ordinal or continuous variable. So, Spearman correlation can be used to look at an association between Age and DV (reporting rho and p-value).

Gender is a nominal variable. If you have two Genders, you can still use Spearman correlation, or see below.

If you have a nominal variable with more than two categories, you have some options, and it can get confusing. You might use either Cochran-Armitage or Kruskal-Wallis to test for a significant effect. Then, for the effect size, you might report epsilon-squared or Freeman's theta. For a more common effect size, you might use the maximum Cliff's delta among paired tests.

Hey Salvatore S. Mangiafico thanks a lot for the detailed response, it's much more clearer how should I proceed.

Regarding the comparison of nominal variables with more than two categories. Can I run the Kruskal-Wallis test? I though this test only worked with ordinal or continous variables IV.

I have some (IV) variables wich are polychotomous variables and are classified as nominal.

The groups variable (DV) are classified also as a nominal variable.

André Soares ,

I'm a little confused.

From your description, I thought you had two DV's, Negative_Word_of_Mouth and Organizational_Reputation. It sounds like these would be at least ordinal. But since they are scales where multiple items were summed, they are being treated as interval, so might as well be treated as interval in the analysis.

Your main IV of interest is Group. But you also have as IVs, Age, Gender, Education. These may be interval, ordinal, or nominal.

My suggestion was to --- as a preliminary step --- is to see if there is a correlation or association between the DV and each of the IV's. For IV's that are at least ordinal or dichotomous, it's easy to use Spearman correlation, p-value, and rho. If an IV is nominal with more than two levels, that's where you may have to measures of association that are a little less common.

Salvatore S. Mangiafico sorry for not explaining correctly, I will try to provide a better clarification.

Yes my DVs are Negative word of mouth and Organizational reputation. I am treating them as ordinal. My main IV as you pointed out correctly is "Group". I did a Kruskall wallis to find if the groups were different regarding this variables and post hoc to find which pairs are different.

Now what I want to check is if the diferences verified among groups, in the previous test are indeed resultant of the stimuli manipulation I did in each group and not due to other variables such as gender, age, academic habilitations, etc. (I want to prove there are no statisticall significant differences among groups regarding this variables).

I was thinking to runnin kruskal-wallis test with the ordinal variables (IV) such as gender, age, academic habilitations and compare if there is a statistically significant diference among groups. But I am not sure what kind of test could I run with the nominal variables, with more than two levels.

What do you recommend?

Do you still consider I should run a spearmans test to check the coorelation between the IV variables and the DVs?

To test for nominal / nominal associations, you can use chi-square test of association (/ of independence), Fisher exact test (including for tables larger than 2 x 2), or some variants of these if there are problems with test assumptions (such as Monte Carlo).

It's fine to check for these differences among the groups. But if you find differences, what does this tell you? Really nothing of interest. For example, if there is a difference in Sex between Groups I and II, does this mean anything? You don't know.

That's why I suggest checking for associations between the demographic variables and the DV. A few reasons: 1) These results are probably of interest on their own; 2) If you do find a difference among your groups in demographics, this gives you some indication of whether this difference is likely to be meaningful; 3) Ideally, you can build a single model with both Group and e.g. Sex, so that the effects of each, and their interaction, can be teased out.