I used sentiment analysis and topic modeling to analyze ~4 million tweets for a study. A reviewer has asked me why I didn't analyze a random sample instead. Is there any reason why it would be better to study a sample rather than the entire corpus?
Reasons for studying a smaller sample:
1.) Responses are not that varied
2.) Constraints on time/money/resources in general.
When machine learning techniques are involved it is common practice to split data set to training one and test one. This enables to verify learning effectiveness on the real data set.
If you did not need to verify the results, just answer the reviewer that such a large set of data did not limit the effectiveness of research in any way, and therefore you had no reason to reduce it.
Many thanks, Jan, Anna, and Ette. These are very helpful answers/suggestions.
Random sampling gave more exposure to our work. They must be thinking that random sampling may give more accurate result. Just go for it. They are more experienced and learned people. So go forward and do as per suggestions . you can also have comparative analysis. it will add more in your research and your efforts will not be get wasted. All the very best
- Badges
- Science topic
- Similar topics
- Philosophy
- Philosophy Of Science
- Reasoning