I collected tweets in 2013 and would like to analyse the topics in them.
you can use social network analysis tools.please check SNA.
useful links:
https://en.wikipedia.org/wiki/Social_network_analysis_software
https://en.wikipedia.org/wiki/Social_network_analysis_software
http://www.kdnuggets.com/2015/06/top-30-social-network-analysis-visualization-tools.html
http://www.butleranalytics.com/20-free-and-open-source-social-network-analysis-software/
First,I think you can use Topic model,and LDA is a method of it.Second,you may use Social Analysis.
Paper explaining theory and methods of computerized text analysis: ( http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=16&cad=rja&uact=8&ved=0ahUKEwjZ64SMrKnMAhWIZCYKHcBPBGQQFghUMA8&url=http%3A%2F%2Ffiles.eric.ed.gov%2Ffulltext%2FED084281.pdf&usg=AFQjCNH4yU9rY0_yJRy1sgv9RvPkkym75A )
Vagueness dictionary suited to content analysis, and link for other WordStat dictionaries:
(http://provalisresearch.com/products/content-analysis-software/wordstat-dictionary/communication-vagueness-dictionary/ )
I agree that Python and R are the best languages to perform text analysis. I suggest you read this publication about a tool to find topics/themes in texts. http://www.sciencedirect.com/science/article/pii/S000437021200077X
I suggest to use Mallet tool,( http://mallet.cs.umass.edu/ ) its free to download and you can use it for many NLP tasks, such as topic modelling and document classification.
Understanding the topic of a tweet, or, in other words, assigning a topic to a tweet would require some bit of text mining - either in the form of clustering or classification. Extra caution has to be taken to perform classification or clustering of tweets since the available text is very short. I am afraid it may not be "fully possible" in excel.
However, if you have a set of topics in mind, and if you can "guess" a list of words that can "strongly" indicate the presence of a topic of the tweets, then you can write excel formula (using substring and match functionalities) to assign topics to tweets (e.g. presence of the word "Grammy Awards" may imply that the tweet topic is Music, presence of the phrase "Wimbledon 2016" indicates that the topic is Sports etc.). However, such hand-encoded rule-based methods often do not work. More over, it is not scalable at all. It is better, as others have suggested, to use tools and techniques from data mining/machine learning domain to get better understanding/assignment of the topics.
You can analyze it using different programming languages like R, Java, Python and some others. but i will suggest you python, because in python for each text processing task has built-in libraries, for part of speech tagging, for tokenizing of sentences and words, for classification and sentiment analysis task. For each and every task you will found builtin libraries.
If you do not have problem, will you please share it with us? because i also doing research in this field.
I have been working with tweets recently. There are tools in "R" and "python". The sentences in the tweets are short. So, I would not suggest topic models like "LDA". The results from LDA doesn't make much sense with the tweets. However, clustering methods like k-means, hierarchical clustering based on term occurrence make much better sense. If you can label the tweets based on the themes, you can also go for some supervised clustering.
The bigger challenge in tweets analysis is the data pre-processing and data cleaning. The language used in tweets are not always grammatically correct form and mostly in slang and hashtags. Based on my experience, I suggest "tm" package in R, for data preprocessing. Also, manually standardizing terms, with higher frequency, might be needed, for example, "because, becoz, cuz" in tweets should all map to one single word "because".
Dear Sir,
Use R language, it has a package for tweet capture. You could analyse them using words of your choice. See Commands explined by Mehdi Zare
Best.
That's correct. The best way to handle these date is the frequency (support) of each word. How any times the word is repeated. Word linked to each topic word can be found using classification. Another opportunity includes the use of classification rules with the form A--> C ( where A is the set of items, and C the class).
- Badges
- Science topic