Hi,

Your question:  Comparing frequency counts between two groups of different sample size, Chi-square?

If your data are in Yes or no type then you have to use Chi-Square test. 

1.  All cells are having >=5, then you can use Chi-Square test. 

2.  If any cell having

You need to use Chi-square, indeed, what u have is a classical application of a contingency table. One sample is about twice the size of the other but they are comparable. If worried about the reliability of results you can use methods such as jackknife to help with your estimations 

http://en.wikipedia.org/wiki/Jackknife_resampling

Thank you, Adrian, for your very useful answer. This is a bit primitive question, but if I want to hand calculate chi-square, what numbers go into these cells in 2x2 table?

Question: Is there a difference between the occurrence of hedging in essays between Group A and Group B? Group A had 50 hedgings, while Group B had 30 hedging.

Chi-square formula: (ad-bc)2 x (a+b+c+d) / (a+b) (c+d) (b+d) (a+c)

                   Hedging       ?

Group A      50 (a)           ?(b)        

Group B     30  (c)           ?(d)

                   80 (a+c)        ?

I am thinking the use of chi-square in not good here as the attributes are unknown. I feel that it is a problem of testing proportions rather than frequencies, as the two sample sizes are different. It is not looking sensible to compare the number 7 with number 35. However, the two proportions can be compared using binomial or normal distribution. 

Thanks, Farah, for your reply. So are you saying that I can use for example t-test to compare 7 and 35?

You can use z-test with pooled estimator of proportion and its standard error. 

Hi,          

Sorry for the belated answer. I will use the initial data, since the data in your answer does not make much sense/is not consistent with data in your question.          

Your contingency table is:

       Hedging no-hedging        Total

Group A        7    a              16      b            23

Group B       35     c            13     d            48

Total             42                 29                 71

= (7*13-36*16)^2(71)/23*48*42*29=

This is your contingency table based on your initial data.

The result is 

Pearson's Chi-squared test

data: .Table

X-squared = 11.6142, df = 1, p-value = 0.0006545

So there is a significant difference at 1%. In my opinion, it makes sense to compare 7 with 35 given that the sample sizes are comparable (23 vs 48), this just shows the highly significant difference between your groups. 

Hi Naoko Taguchi,

Simply, you can use The Pearson chi-squaret est, thus I disagree with few previous authors about t or Z-tests.

The result is as follows:

χ2 value=  11.61

df = 1

P value= 0.0007

Thereby, there is highly significant difference among examined students regarding hedgings  factor. 

Good Luck

Thanks, Adrian. But the problem is that there is no absolute value for the frequency of hedging. So for example in your table Group A produced a total of 7 hedging and Group B produced a total of 35, but it's not that 7 people in Group A produced hedging and 35 people in Group B produced hedging. It's not about frequency of people, but it's about frequency of this linguistic feature (hedging). So should I go with methods for comparing two means?

I don't know the difference between the Pearson chi-square and regular chi-square.

Hi, I think it is the same regular and Pearson chi-square, but I am not entirely sure. Anyway the method is tried, tested and true, and you can find scores of analyses done with it.

You have two options, to count the people and to count the events. For the first measure you analyze people with hedging vs the others, and for the second you analyze hedging events vs. non-hedging events. Then you apply the chi-square on both. You can compare the means for similar measures, either people or events, but in my opinion you can use chi-square.

Thanks so much for your reply, Adrian. I'm sorry for keeping asking you questions, but how do I count non-hedging events? If Group A has three students, and each produced 10 hedgings in their essays, frequency will be 30 (3 students x 10 hedging). But how about non-hedging events? if the frequency is based on the appearance of particular phenomena and there is no absolute value, I'm not sure what the frequency looks like.

Hi Naoko,

I think it is a problem of comparing two groups A and B and we have to decide which measure will be useful to compare, as the sample sizes are different.

Group A had 23 students and they had a total of 7 hedging (Average is 7/23=0.30 hedging per student), whereas in  group B there were 48 students with 35 hedging (Average is 35/48= 0.73 hedging per student). It looks a significant difference among the two groups which can be confirmed using a two sample test for proportions. 

Remember that even if you are using a chi-square test, the hypotheses are formed in the form of proportions....

Good Luck

Thanks, Farah. I will use two sample test then (e.g., z-test, t-test).

Then the only option is to consider the number of students for which hedging events occur and  compare them with the students for which no such events occur.

Thanks, Adrian!

You can use Z-test for 2 population proportions

Hi,

Your question:  Comparing frequency counts between two groups of different sample size, Chi-square?

If your data are in Yes or no type then you have to use Chi-Square test. 

1.  All cells are having >=5, then you can use Chi-Square test. 

2.  If any cell having

I have a follow up question. How about if i have more than two groups, let say, Group A, B, and C. Can I use G test?

i know my answer is a bit late, but i need to ask you what statistical procedure you finally used? i think using a log-liklihood calculator is the best answer. actually, i have the same problem.

For more than two groups, test for several proportions can be used. Ultimately, it is the Chi-square test of independence between the two attributes.

Regards

For two-sided tests there are some recent studies on the topic considering low proportions by I.Campbell:

(1) Where all expected numbers are at least 1, analyse by the ‘N − 1’ chi-squared test (the K. Pearson chi-squared test but with N replaced by N − 1).

(2) Otherwise, analyse by the Fisher–Irwin test, with two-sided tests carried out by Irwin’s rule (taking tables from either tail as likely, or less, as that observed).

for more you can find here

http://www.iancampbell.co.uk/twobytwo/twobytwo.htm#hhhhh

I personally needed 1-sided test so 1-sided fishers exact and binomial distribution test for low frequencies and z- or t- test for other cases worked for me. You can see more detailed information here https://towardsdatascience.com/spotting-conversion-rate-drop-with-two-sample-hypothesis-testing-using-e-commerce-monitoring-24542ada6122

How about relative frequency as the sample sizes are different for the two groups (as far as we are not going to compare averages but simply frequencies)?