I want to get data from social sites like Facebook, twitter, instagram etc. about my company (interest) etc. and later on I want to apply some analytic on that particular data like reports, security, reply message, post status etc.
Is there any open source tool available for this particular purpose ? I go through with search and found that, we can retrieve data from Twitter Hashtags '#' for this purpose, but for this we have to make a website in which that data will be published. Is there any other solution for this particular task ?
Hi Asad,
to capture and analyze tweets you can use the open-source DMI-TCAT toolset, available at https://github.com/digitalmethodsinitiative/dmi-tcat.
To capture Facebook data you can use the online tool Netvizz, available at http://apps.facebook.com/netvizz
The accompanying academic papers can be found at https://www.researchgate.net/publication/263203930_Programmed_method_Developing_a_toolset_for_capturing_and_analyzing_tweets and https://www.researchgate.net/publication/262160375_Studying_Facebook_via_data_extraction
Best,
Erik
Article Programmed Method: Developing a Toolset for Capturing and An...
Conference Paper Studying Facebook via data extraction
Asad, You can build your own customized crawler. In 2009 I built and published the source-code and algorithm for one such harvester. It accesses public sites and can archive publicly visible data in a nice machine-readable format. I use it myself, to develop an organically growing dataset of 'hard to get' bid-ask prices on US traded ETFs. Thereafter you can do your analytics on it.
Additionally you can contact these platforms and a number of them will be happy to share their API with you, from which you can pipe the relevant data. Matlab provides such functionality too.
Not sure if this addresses part of your query. Cheers!
http://www.emeraldinsight.com/journals.htm?articleid=1780840
Article An automation algorithm for harvesting capital market inform...
Hi Asad,
to capture and analyze tweets you can use the open-source DMI-TCAT toolset, available at https://github.com/digitalmethodsinitiative/dmi-tcat.
To capture Facebook data you can use the online tool Netvizz, available at http://apps.facebook.com/netvizz
The accompanying academic papers can be found at https://www.researchgate.net/publication/263203930_Programmed_method_Developing_a_toolset_for_capturing_and_analyzing_tweets and https://www.researchgate.net/publication/262160375_Studying_Facebook_via_data_extraction
Best,
Erik
Article Programmed Method: Developing a Toolset for Capturing and An...
Conference Paper Studying Facebook via data extraction
Thanks to all of you, especially Erik Borra .
Your answers are very helpful
If you want historical Tweet data, you would have to contact a company that sells them, as the historical tweet API is limitted to a couple of days.
for example http://discovertext.com (they give free data from time to time).
If you want data on the flow for Twitter, the Twitter Streaming API is fairly easy to use. If you program in java, here is a library that allows to easliy get you data from the API
http://twitter4j.org/en/index.html
For Facebook, the best solution would be to develop Facebook app and have some users authorize it.
https://developers.facebook.com/
number of crawlers/harvesters are available to address this issue, now it depends upon your work, if micro-blogs are to be investigated, then I recommend: twitter streaming API's as mentioned by Omar Jaafor . However, if reviews/blogs/news are your target for analysis, then python-based BeautifulSoup is well suited, available at:
: http://www.crummy.com/software/BeautifulSoup/
For more help, look at the following works:
1. https://www.researchgate.net/publication/283318703_Detection_and_Scoring_of_Internet_Slangs_for_Sentiment_Analysis_Using_SentiWordNet?ev=prf_pub
2. https://www.researchgate.net/publication/284344945_Context-Aware_Spelling_Corrector_for_Sentiment_Analysis?ev=prf_pub
3. https://www.researchgate.net/publication/283318830_Lexicon-Based_Sentiment_Analysis_in_the_Social_Web?ev=prf_pub
Article Detection and Scoring of Internet Slangs for Sentiment Analy...
Article Lexicon-Based Sentiment Analysis in the Social Web
Article Context-Aware Spelling Corrector for Sentiment Analysis
Actually, I am working on Elastic search tool, ELK stack (Elastic search, Logstash and Kibana) , so i want to fetch data using these tools, if is there anyone have an experience on work on them, do inform me
Regards:
Asad Masood
- Badges
- Science topic