You can also extract Twitter data with NodeXL. It's a plug-in for MS Excel and - from what I've heard since I haven't used it myself - seems to be quite user-friendly: http://nodexl.codeplex.com/

You can also extract Twitter data with NodeXL. It's a plug-in for MS Excel and - from what I've heard since I haven't used it myself - seems to be quite user-friendly: http://nodexl.codeplex.com/

I suggest you to explore ScraperWiki https://scraperwiki.com/

You just have to create a free account and start downloading Twitter data to excel or csv files. There is more infor here: https://scraperwiki.com/tools/twitter

The simple python script to dump data stream from twitter "as is" - https://github.com/uwescience/datasci_course_materials/blob/master/assignment1/twitterstream.py. Readme for this script is here - https://github.com/uwescience/datasci_course_materials/blob/master/assignment1/README.html.

Sana,

If you haven't done so, I'd suggest contacting the creators of 'Mappy Health' at http://mappyhealth.com/

Brian Norris, Mark Silverberg and Charles Boicey have been mining tweets for health information. Their contact information is available at:

http://mappyhealth.com/about

Hope this helps.

Due to my experience, it is better for you to learn bit of programming, and using either nodejs or python to get data from twitter.

Sana, two thoughts. First, a lot of the comments you're getting are on means to collect live streams. Another option is to locate an existing corpus of Twitter data such as the Internet Archive.

https://archive.org/details/twitterstream

http://archive.org/search.php?query=collection%3Atwitterstream&sort=-publicdate

Be aware that you'll need a lot of disk space to start downloading these archives.

Second thought, regardless of your source make sure you understand how much coverage you're getting. Unless you're getting a full "firehose" of twitter data from a reseller such as http://gnip.com/, you will not be getting all tweets in a time period. Check out http://allthingsd.com/20101110/twitter-firehose-too-intense-take-a-sip-from-the-garden-hose-or-sample-the-spritzer/ for some more details. The Internet Archive is the "spritzer" level of tweets, or about 1% of all tweets.

I'm aware of some research groups having full firehose access, but only because their funding supports the cost. For example, I think Arizona State's TweetTracker program has full firehose. http://tweettracker.fulton.asu.edu/ But they're unable to share data due to their license with Twitter.

Finally, getting back to live sampling of twitter. If you collect your own twitter data you can set your own queries, which means you can get better value from the 1% spritzer stream. You'll still be getting 1%, but the odds of the tweets you get being relevant goes way up.

Same question , but concerning facebook, how can i extract information of member fun for a page, and after i can explore these information to making some profiling and clustering

@Ben. My research area is the social business intelligence, the objectives are to explore unstructured informations in social medias, to construct a business intelligence system.

my questions; these paterns and APIs are for free or must purchase them

@Adil. The access to that APIs are for free to everyone. In some cases, you just need to create a developer account ( like in twitter)

I think the Internet Archives repository mentioned earlier, might be all right for your needs:

https://archive.org/details/twitterstream

Otherwise, the following python code and instructions should hopefully give you a gentle introduction to collecting data from the Twitter stream:

https://github.com/computermacgyver/twitter-python

I found the following publication very helpful:

Kumar, Shamanth; Morstatter, Fred; Liu, Huan (2013): Twitter Data Analytics. New York, NY, s.l: Springer New York (SpringerBriefs in Computer Science).

Take a look at: http://tweettracker.fulton.asu.edu/tda/

On an irrelevant (but not so much) note, you need to consider the legal ground upon which you will rely the processing of private data, based -of course- to the relevant legislation in your place.

NodeXL (nodexl.codeplex.com) is a great (and free) tool for colelction od Twitter data directly to your Excel, and for network analysis. It shows you the tweets as well, which can be helpful for your analysis.

For public health type issues trending on Twitter I suggest the following http://nowtrending.hhs.gov

R language package twitteR can be used for collecting twitter data. QVsouce is an easy tool for crawling twitter.

I found this resource from the Canada-based Social Media Lab. It contains research analytic tools for various social media including Twitter. Feel free to check it out, and let me know if it was any help.

http://socialmedialab.ca/?page_id=7801

@Henry Boachi Thankyou for the link. It is helpful

Welcome. I've also found a relatively crude way of collecting Twitter data. If I am following a particular hashtag, I type it in the search function on Twitter at the end of a particular day. Twitter gives me all the tweets using that hashtag for the day. I then highlight everything and save/print it as PDF. It's crude but it works :)

Hi I am in the same situation i want to take tweets from advance search a year back for example from 01 Jan 2014 to 28 Feb 2014 but i dont know how to save them. I dont want to save them in .pdf  because I can not edit the data. Did you find a solution?

Hi Vasilis,

Try Twitter TAGS Explorer, which lets you archive all your Tweets into a spreadsheet or .csv file:  http://tags.hawksey.info/

I have a similar question, how to download random tweets for specific time period?

I want to download random tweets from Twitter for specific time period (of two years 2011-2013). I have tried using statuses/sample API, but couldn't specify the time period.

I'm a Communication professor and used to be in the same boat five years ago when I started learning how to download tweets. There are a lot of new moving parts for many people -- relational databases, API, JSON, XML, etc. I learned a lot from online forums like StackOverflow, so I'm trying to give back and have created a series of tutorials for the complete coding beginner. Take a look and let me know if they're helpful:

http://social-metrics.org/tutorial-list/

Happy coding!

Thank you  Gregory for the link to your blog and the tutorials you have shared. I look forward to looking in to this! 

DFor anyone interested in NodeXL Marc Smith provides an overview in this recent video he shared http://bit.ly/1FGWq07

Also I would recommend looking at the Social Media Research Foundation web pages 

http://www.smrfoundation.org/nodexl/

http://www.smrfoundation.org/nodexl/

You're welcome! I would say that if you have a PC and you only need a static, one-time download, then something like NodeXL might be the way to go (I've never used it). If, on the other hand, you'll be doing repeated downloads (downloading tweets over time, for instance), then you'll need to get into using a database. This will avoid duplicates, among other benefits. That's where my tutorials come in -- showing how to use Python to pull the tweets and put them in the database. 

Here is the complete solution of your requirements:

Hootsuite

Viralheat

https://www.viralheat.com/

www.hootsuite.com

TAGs remains the best as it is easy to use and can work even if you are offline

Netlytics is my absolute favourite now, and keeps on working while you're offline too.  It's good for getting Twitter, Instagram, Facebook and Youtube data too: https://netlytic.org/home/

Also check out the Digital Methods Initiative tools: https://wiki.digitalmethods.net/Dmi/ToolDatabase

I wrote a webapp to keep following topics through Twitter, based on the hashtags. You can use it for educational purposes. Results are downloadable as json or csv and images go in a seperate directory. Of course, you plug in to the stream from the moment you start following (and then 7 days backwarck, the maximum the Twitter Api allows), but the search is repeated on regular basis through cronjob and saved in DB so you can follow for a longer period.

Hi,

You can get twitter data from Podargos Data. It can provide twitter historical and realtime data. In additional, it's able to help you retrieve other mobile App data of social networks and e-commerce, and supports for nearly 100 languages. All you have to do is to tell them your needs. I hope this information can help you.

Podargos' website is http://www.podargos.com

Here is the data sample: 

http://goo.gl/dRxyow

Please refer my paper

R users have developed a number of tools that access Twitter data and process it - see blog at: http://www.rdatamining.com/docs/twitter-analysis-with-r for details.  R isn't everyone's idea of good analytical software but a lot of help exists in the user community if it causes problems.  Hope it helps! 

Thanks for the tip Vincent

http://DiscoverText.com is free for 3 days with Search API and Gnip PowerTrack 2.0 access. Sifter (http://Sifter.texifter.com) offers three free historical Twitter estimates per day and the only self-serve web interface for purchasing access to the complete (undeleted) history of Twitter using Gnip's Historical PowerTrack 2.0.

Use twecoll. https://github.com/jdevoo/twecoll

You can extract tweets and build graphs files. These can be open with Gephi

(You can get twitter data from Podargos Data.) 

i treid Podargos  they are sooo expensive 

I think that the easiest way to collect datasets from Twitter is using R Blog:

Example of R packages:

http://www.r-datacollection.com/blog/TwitteR2Mongo/

try using a tool called selenium https://www.seleniumhq.org/, an open source web crawler.

I've tried it for crawling data from OSNs, mainly twitter and facebook, with a data scale in thousands of profiles, pages, comments, etc. and it worked perfectly

good to see no mention of the ethics - (being sarcastic here!)

GDPR - twitter handle as personal identifying information (as well as ISPs) - lets just go and scrape anything we can privacy be dammned - €20 million fines anybody - and most reputable journals will want to know why no ethics approval