How can I collect tweets produced over some period of time, say in two weeks for few specific users in twitter for example? Is there a microblogging website with over a million users that will let me collect anonymous data of its users posts/tweets easily with its api? I'm more interested in only textual contents.
Hi Hassan, you can request the microblog track of TREC 2011 dataset provided by NIST, the corpus contain mearly 16 millions tweets collected in Two weeks between January 24 and February 8th, and you can extract the all corpus using the twitter corpus tools that use the Twitter API (https://github.com/isoboroff/twitter-corpus-tools)
Actually i think the storage part would be the easier side, there are NoSQL data stores that can store and scale up to high/ample levels/types of data and there are processing platforms like Hadoop and Disco (Map-Reduce implementations) and other analytic guess one just needs to keep the bear fed.
The Twitter API will let you do that no problem. You will just need to have an OAuth key to let you access specific users.
Hi Hassan, you can request the microblog track of TREC 2011 dataset provided by NIST, the corpus contain mearly 16 millions tweets collected in Two weeks between January 24 and February 8th, and you can extract the all corpus using the twitter corpus tools that use the Twitter API (https://github.com/isoboroff/twitter-corpus-tools)
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining