Hello friends, I'm looking for criteria for minimum number of words in keyword and collocate testing. I have seen work addressing potential problems with very large corpora (for both log likelihood and chi-squared tests), but I can't seem to find anything on corpus size minimums. Specifically I'm interested in log-likelihood and MI-score. Thanks for your help.
Perhaps p. 979-980 in this paper might help: Brysbaert, M., & New, B. (2009). Behavior Research Methods, 41(4), 977–990. Cheers.
I have performed experimental studies on evaluating the performance of MI and log-likelihood using small and very large corpora. The measures, especially the log-likelihood, showed good performance on the small corpus (of about 80,000 words). However, their performance dropped when applied to a very large corpus (of about 50,000,000 words). You can find the study here:
https://www.researchgate.net/publication/268518398_An_Empirical_Study_On_The_Holy_Quran_Based_On_A_Large_Classical_Arabic_Corpus
Article An Empirical Study On The Holy Quran Based On A Large Classi...
Wei Ping, Mohammad, thank you both for responding.
Maha, thank you so much for both answering my question, but also for sharing your work.
1. From your "Study on the Holy Quran" paper--it look alike the first run was done on a relatively small Quranic corpus, and the 2d run was done on the relatively large KSUCCA, and while MI and LL are adequate for the smaller corpus (they work in conditions of data spareness), they perform poorly on large corpora. In your reply here you mention the corpora sizes, however in the paper I don't see a size for the Quranic text. Is the issue that you studied the Quran as a single text, and it is common knowledge that the Quran is 80k words? Forgive me if I'm ignorant of common cultural knowledge in your part of the world.
2. The Refined Dice paper is a fascinating study btw, and using Refined Dice to extract syntagms from text seems like an important advance.
3. I was excited to read your theoretical framing. I'm a discourse analyst and sociolinguistic rather than a computational science researcher, so I'm very concerned with the linguistic assumptions underlying computational approaches to language, because it impacts method and design so strongly. In articulating an empirical approach to real world discourse, you point out the insufficiency of Chomskyian/UT Grammar theories that treat language as an a priori rules-based system. Whereas empirical analysis of actual discourse show that structure in language use is an emergent, temporal, contested process (Hopper, 1987. "Emergent Grammar." Proceedings of the Thirteenth Annual Meeting of the Berkeley Linguistics Society pp. 139-157). I only know of a few computational linguistics researchers who explicitly address this, and draw on emergence to improve their empirical work ( e.g. Cimino & Vaglini, 2013; Vlas, 2012), so it was encouraging to read your paper.
Thank you William.
Regarding your question, yes, the Quran is the holy scripture of Muslims and it is considered to be the words of God. It is composed of a single text of about 80k words.
Thanks Maha :) I do know what the Quran is--I wasn't sure if "Quranic text" meant something distinct, like text about or associated with the Quran.
- Badges
- Science topic
- Similar topics
- Social Psychology
- Attitude