Pooja,

I doubt I can shed more light on this situation than the following paper by Mehta and Hilton (1993).

Article Exact Power of Conditional and Unconditional Tests: Going Be...

A PDF of this paper can also be found here:

https://www.jstor.org/stable/pdf/2685184.pdf?casa_token=kkPIA_QWr4gAAAAA:be2wA2CsB92b4Wr9tkFuNeF6NzblGbqu6GSnYtZkTom4akwcBArBwqIBweptBjevgQC5Y0FZlclJdboNmk71pmyEaa3WRVfK9sYmVNjtomGhbRWByg4UHg

You might also find older, related RG discussion (in following address) useful.

https://www.researchgate.net/post/Alternatives_of_Fishers_exact_test_for_more_than_2_groups2

Best wishes,

Caleb

Do you have an implementation for Barnard's test for a 4 x 4 table? If not, that may decide the question.

In my experience people tend to ignore the fixed-margins assumption of Fisher's exact test. I don't know if this is because they don't know about it, or if there's a more informed reason.

I believe Fisher's test should be more conservative (less likely to report a low p-value), so it may be an acceptable approach from that perspective.

I found the following (brief!) discussion helpful. https://stats.stackexchange.com/questions/234043/barnards-test-2x4-table-in-r

I also looked in Agresti, 2013, Categorical Data Analysis, 3rd. He's got a few paragraphs dedicated to this question, but honestly, I'm not sure there are many take-aways. He does note that Bernard "withdrew" his test in favor of Fisher's test. There is a later test by Boschloo, that necessarily reports a p-value less than or equal to Fisher's.

Thank you , i didn't realise that in R barnard's test for 4 by 4 matrix is not possible. I think i will have to stick to the Fisher's exact test. I would try Boschloo test.

I forgot to add/ask: Given that you have a sufficient sample size, wouldn't Pearson's X2 test be appropriate? Or, a re-sampling method if sample sizes are smaller (I think this is suggested in the RG question I linked)? @Pooja @Salvatore

https://www.uvm.edu/~dhowell/StatPages/More_Stuff/Chi-square/Contingency-Tables.pdf

@Caleb for pearson's chi square test one need to have some expected value right? In My experiment i score the behavior of the animal

(which vary a lot, even if everything is kept constant), hence I can't have an expected observation. Can I still apply chi square test?

No, the expected values for the chi-square test of association are calculated internally by the test. They are not like the theoretical values of the chi-square goodness-of-fit test. Just confusing terminology.

The only thing is, you should check the expected values, say, that all are greater than 1 and not more than 20% are less than 5.

Ok, thank you @Salvatore