I want to normalize two linguistic corpora; however, I have no idea whether I need to normalize the corpora per 10,000 words or 1,000,000 words. How can I decide about these two? Why do we use 10,000 and 1,000,000 words, but not 100,000, for example?
Thank you in advance.
The bigger, the better a corpus is. Because 10,000 words is the size of a research article (usually journals require between 7000 and 9000 words). Why a big size for a corpus? A big size yields more representative outcomes. Nowadays corpora count billions of words.
Thank you very much Dr. Maalej. Each corpus used in my study contains around 25,000 words. So, I think 10,000 would be appropriate.
Best regards,
If the two corpora are 25,000 why go back to 10,000 words, which is a tiny little file of a corpus. As I said in my previous post, the bigger the better.
Dr. Maalej,
The two corpora together contain around 50,000 words. At the time being, I cannot increase the corpora. So, concerning the number of words in these two corpora I think that "per 10,000 words", would be good. My problem is whether I have to use "per 10,000 words" or "per 1,000,000 words" while my corpus is small. What is the rational behind using "per 10,000 words" or "per 1,000,000 words" while normalizing the different-sized corpora?
Thank you so much Dr. Etuk. My problem is exactly what you mentioned: Do I have to use "per 10,000 words" or "per 1,000,000 words" while my corpus is small? What is the rational behind using "per 10,000 words" or "per 1,000,000 words" while normalizing the different-sized corpora? Who has decided about these numbers? Do you know any articles in which different constant numbers for normalization have been proposed in addition to the reasons? I know the reasons for using 0.01 or 0.05 as levels of significance, but nothing about using "per 10,000 words" or "per 1,000,000 words"?
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS