Hi Team,
Good Day.
I am working on a text mining assignment and have built the document matrix using the tm package.
Now I need to run findAssocs from my dtm with some word say 'like' with a correlation of 0.70 but as far as i have been researching it tells it this function is only viable when we have more than 1 doc however in my case i only have 1. So please suggest if there is an alternate to this issue.
Kindly advice.
Hi,
you could split your data differently. E.g. consider each paragraph/section/sentence etc a "document".
Hi Gard,
As i have done all the data cleaning and have created a corpus and then dtm hence all my data be it in rows or column or sentence is not into one single row.
Please see my code below and suggest:
x
Hi Shivi,
as far as I can see, your code is passing a single vector into the Corpus function, and from there into the DocumentTermMatrix() function. So essentially, you're creating a matrix with 1 row and as many columns as there are terms. Hence, no correlations.
It looks like your code is merging together a bunch of reviews. So one option would be to consider each review (i.e. each row in the .csv file) a single document, instead of merging them. That way you could find associations between terms across multiple reviews.
Yes, this was a solution I also had been looking for. I can treat each respondent as a book perhaps.
Don't get confused by the term "document" in the context of bag of words text mining, which is the type of analysis you are using when you create the term-document matrix.
This analysis considers the presence of each word or groups of words, a unique feature of each document. Thus, each document is comprised of the smallest unit of text that makes sense to process as a stand-alone unit. Think of responses from a open-ended survey, tweets to a particular hashtag, paragraphs, etc.
I hope this clarifies why you are asked to do this break down of your text file into paragraphs for bag of words analysis searching for associations. Be aware also that associations are useful when looking to answer a question in relation to the usage/appearance of a frequently used word, whether it is expected or not for the general subject of the main corpus of documents. That's why usually frequency analysis comes before association.
- Badges
- Science topic