I am currently doing my undergrad in Marketing Communications and Public Relations, doing a dissertation. I would like to conduct a content analysis of YouTube comments, I am looking for any advice on how to calculate how many comments I will need to create a well established overview for my 10,000 paper.
Any help with be appriciated! Thank you so much.
There isn't really any standard for this kind of sample, so I suggest you approach the issue from a different direction: How many comments could you reasonably expect to code, given the amount of time available? This first step in answering this question would be to estimate how many items you can code in an hour.
Dear Amber Ellis , determining a reasonable sample size is not so much a question of your topic and the medium (in your case YouTube) but rather a) (as David L Morgan already said) of the time you have available and b) of the methods/tests you want to apply to your data and the coefficients that come along with it and should per convention of your discipline be reported.
I for instance also have a background in communication science where it is for classical statistical analyses (aka hypothesis testing) quite usual to rely on the reporting guidelines privided by the APA: https://apastyle.apa.org/products/publication-manual-7th-edition
or
https://psych.uw.edu/storage/writing_center/stats.pdf
If you know, which tests you want to run, which metrics you want to report, and what values of these metrics would be the mimimum to expect, so that you would call your results 'good' or 'reliable' results, you could use tools such a GPower to perform a priori power analyses to determine the minimal sample size:
https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower
Usually, you should be good with ~100-200 coded examples.
If you however would want to carry out analyses that involve automated-content analysis methods that feature supervised machine learning algorithms, you might well need several hundrets or even some thousand examples more to archive good results (in terms of measures as the F1 score etc.). This is nothing that can be exprected in a student paper unless you manage to find a YouTube dataset that is already labeled.
If on the other extreme, you want to select a more qualitative approach, a thorough analysis of 15 or 20 comments might serve you just as well.
Anyways, good luck with your thesis!
Hi Amber,
Once you have selected the videos, you have to take a sub-sample of comments within each video. One possibility is - within a given video - to randomly select the first comment and then select successive comments at intervals of size "k". This interval k depends on the number of comments to be selected in each video. This design is called systematic sampling.
Because we assumed a similar number of comments in each video, the size of each sub-sample of comments will be the same for all the videos. You can retrieve the number of comments in each video when you collected the data. If you use NodeXL or Netlytic, for example, they'll let you know how many comments have been collected.
After the sample size is calculated, you can randomly select the 10% of this sample for the intercoder reliability test.
- Badges
- Science topic
- Similar topics
- Comment