If you have expected counts below 5, Fisher's Exact Test is an alternative to a chi-square test. The test is usually included in general statistical software and many online calculators exist if you are testing independence (for example, see http://www.danielsoper.com/statcalc3/calc.aspx?id=59

or

http://vassarstats.net/fisher3x3.html)

If you have expected counts below 5, Fisher's Exact Test is an alternative to a chi-square test. The test is usually included in general statistical software and many online calculators exist if you are testing independence (for example, see http://www.danielsoper.com/statcalc3/calc.aspx?id=59

or

http://vassarstats.net/fisher3x3.html)

If your data is above 2x 2 matrix and lends itself to collapsing you can do this to avoid numerical counts below five (so merge columns or rows to allow additions of small values.

alternatively you can move to correspondence analysis whic will also allow you to look at multiple variables (more than two, as with chi-square) I.e. Multiple correspondence analysis, which does not have same low count conditions as chi-square

Re Jason's suggestion of using a "G-test" (i.e., the likelihood ratio chi-square), I have some relevant commentary in an old set of notes on categorical data analysis--see section 2.10 at the link below (and note that I use both L2 and G2 to refer to the test statistic for the likelihood ratio chi-square).  In a nutshell, I ended up siding with Alan Agresti, who argues that Pearson's chi-square statistic works better than the likelihood ratio chi-square when expected counts are low.  (But perhaps there has been more recent work relevant to this question.)

@ Huda:  What are the minimum and maximum expected frequencies for your 3x3 table? 

http://www.angelfire.com/wv/bwhomedir/notes/categorical.pdf

Many thanks for all.

 Fisher's Exact Test on a 2x2 table only.

The traditional Fisher's exact test is only computationally realistic for 2 x 2 tables, but the Freeman-Halton extension to Fisher's test generalizes to tables larger than 2 x 2 and takes seconds to compute. The two links I posted above calculate Fisher's test using the Freeman-Halton extension for 3 x 3 tables and returns p-values almost instantaneously. 

Hi  Bruce;

Some categories have be joined together. Now, I have 2x3 table. The minimum and maximum expected frequencies for a 2x3 table are 4.3 and 9.9.

Here is a pretty standard rule of thumb regarding the validity of Pearson's Chi-square for tables larger than 2x2.  Pearson's Chi-square will be fairly valid if:

• Each observation is independent of all the others (i.e., one observation per subject); and
• "No more than 20% of the expected counts are less than 5 and all individual expected counts are 1 or greater" (Yates, Moore & McCabe, 1999, p. 734).

So if only one of the six cells has E < 5, Pearson's Chi-square should work quite well.

HTH.

Bruce, but I have 2 cells (33.3%) have expected count less than 5 (which is greater than 20%).

Collapse categories to 2x2 followed by Fisher's exact.

or

Freeman-Halton extension if greater than 2x2, but this may be limited by total N - the exact tests involve factorials which can take a lot of CPU.

I will try to build a model, such as loglinear model, and give a extremely small number to the cell with zero frequency.

Fisher's exact test

can i use chi-square test if my observed value or frequency is less than 10?

What I can do if the frequency of one variable is zero?

I think all the answers provided above are valid, but I prefer collapsing the table to avoid the small expected numbers in the cells, i.e combining categories in a meaningful way.

I do the same as Anwar.

reducing degrees of freedom or merging the rows or columns till you see atleast 20% of the cells have expected count more than 5 and carryout chisquare test

ANOVA is a good alternative as it is a quantitative as well as qualitative test.

The Freeman-Halton test is an extension of Fisher Exact for 3 x 3. You can do the calculations at this site.

http://vassarstats.net/fisher3x3.html

In this discussion two different questions are mixed together. One question is what statistic one should use. Another question how one should calculate or estimate the distribution of the statistic that one has chosen to use.

For any sample size one can use the following method. Use the G2-statistic and estimate the p-value by simulation.

If one wants to avoid simulation and there are few cells one can always approximate the distribution of the G2-statistic by a chi-square distribution. This holds even if some of the counts are low. If there are many cells and 80% of the cells have at least 5 counts and all cells have at least 1 count, then one can use the chi-square statistics and approximate its distribution by a chi-square distribution.

Hi Peter. As mentioned earlier in this discussion, Alan Agresti argues (in his book, Categorical Data Analysis) that Pearson's chi-square statistic works better than the likelihood ratio chi-square (G2) when expected counts are low.  Do a Ctrl-F search on to find the relevant posts. HTH.

Ch is quart test is ideal. If more than 20% cells have expected frequency less than 5 then pooling row or column can be done. In case of 2x2 tables fisher exact test can be done.

Hi Bruce You have to distinguish between the choice of statistic and the calculation or estimation of the distribution of the chosen statistic. Agresti's arguments is only about the approximation of the distribution of the chosen statistic by a chi-square distribution. If one simulates the distribution of the statistic Agresti's arguments are irrelevant.

If the number of bins is small the G2 statistic actually has a distribution that is closer to a chi-square distribution that the distribution of the chi-square statistic. I have some papers stressing this point:

July 2012 (with G. Tusnády). “Information Divergence is more χ2 - distributed than the χ2 -statistic”. In: International Symposium on Information Theory (ISIT 2012). IEEE. Cambridge, Massachusetts, USA, pp. 538–543. url: http://arxiv.org/abs/1202.1125.

June 2014. “Mutual information of Contingency Tables and Related Inequalities”. In: Proceedings ISIT 2014. IEEE, pp. 2474–2478. url: http: //arxiv.org/abs/1402.0092.

2016, "Bounds on tail probabilities for negative binomial distributions". Kybernetika 52 no. 6, pp. 943-966, DOI: 10.14736/kyb-2016-6-0943.

Sorry Peter, I was in far too big a rush when I read your post and missed that you were proposing to simulate the sampling distribution (even though you said so clearly). Mea culpa.

The output from analysis of categorical variable is a p value indicating a YES or NO response to statistical significance. Statistical significance can also be derived from OR or RR with 95percent confidence interval. This is not dependent number in a cell. This is situation in outputs from logistic regression analysis for novitiate dependent variables

........i mean bivariate dependent variable

Bootstrap techniques can also be used.

Fisher exact test is best alternate to chi square test

Hi Muhammad,

Yes, we can use Fisher exact test with 2 X 2 tables , but if we have two categorical variables both of them have 3 levels  and the (33.3%) have expected count less than 5. What is the alternative test.

Regards,

Huda A. Rasheed ,

well, you can search R package relating to your problem. it may helpful to identifying appropriate test

I generally use Kolmogorov-Smirnov for two groups, and many columns.

http://www.physics.csbsju.edu/stats/KS-test.n.plot_form.html

There are exact tests comparable to the Fisher for more than two columns.

http://www.quantitativeskills.com/sisa/statistics/ord2.htm

Try collapsing your cells, if meaningful, to a smaller table. If your table is now 2 X 2, use Fischer's Exact Test. If it is bigger than 2 X 2, your Chi-Square test may now be valid.

SPSS v25 can do Fisher's exact test for table larger than 2 X 2

You can use the Freeman-Halton test (see for example https://stat.ethz.ch/pipermail/r-help/2011-April/276712.html) or you can consider a new test proposed in the article Zhang, Q. Independence test for large sparse contingency tables based on distance correlation, Statistics and Probability Letters, 148 (2019) 17–22.

Try with G-test

https://en.wikipedia.org/wiki/G-test

Or Fisher's exact test

https://www.jstor.org/stable/2288652?origin=crossref&seq=1#page_scan_tab_contents .

The G-test is an asymptotic test, and in my opinion, in the case considered, it can not be used (likewise the chi-squared test) .

We should do Yates correction or Fishers exact test

Alicja Jokiel-Rokita Thank you.

Waiting for Solution!!!

What is the alternative test of Chi-square, if there exist one cell has Observed frequency is Zero? can fisher exact test is applied?

Also which test to measure the association between dichotomous variable vs variable with three labels.

Sudip Poudel , the rules of thumb regarding Chi-square tests are concerned with expected frequencies, not observed frequencies. Here is a summary of the common rules of thumb:

• https://sites.google.com/a/lakeheadu.ca/bweaver/Home/statistics/notes/chisqr_assumptions

Notice that when the marginal totals are not fixed in advance, the N-1 Chi-square is often a better choice than the Fisher-Irwin test (aka., Fisher's exact test). See Ian Campbell's Statistics in Medicine article for details. His summary of it can be seen here:

• http://www.iancampbell.co.uk/twobytwo/background.htm

HTH.

What matters is the expected counts not the observed. If the counts in any cell is less than 1 or 25% of the cells has expected counts less than 5, then the Fisher's Exact test is appropriate. However, if the table is more than 2 by 2 it would be better to collapse the table so as to increase the expected counts in the cells

Thanks Anwar for your answer. But if we can't collapse the table so as to increase the expected counts in the cells, what is the best alternative test.

Regards,

Dear Huda,

The Fisher's exact test.

Huda A. Rasheed

If the order of the contingency table is of higher order you have to pool either the rows or columns to make the table to have frequencies in all the cells. If it is a 2 x 2 table and if table has more than 25% of the cells exp frequency < 5 then use Fishers exact test,

Ravishankar Suryanarayana

If the row categories are different. Also the categories of columns, so you can not merge rows or columns. What is the most suitable test that can be used instead of chi-square test?.

It is advisable to use a Fisher test of Independence.

Instead of traditional selected-response or constructed-response tests that look for discrete facts or knowledge students recall in a standard way, students can apply knowledge in alternative, novel ways. Writing poetry in a language arts class, performing in a play in a theatre class or a mock-trail in a government class are alternative assessments. These performances are assessed with rubrics, which are also used to give feedback to students and stakeholders.