Hello Kalliopi and others,

Raven's Eye (https://ravens-eye.net) automates the analysis of a number of types of textual natural language expressions, including transcribed survey responses and interviews, as well as written documents, books, and other texts.  Brief examples of it applied to U.S. political party platforms, and Nietzsche's Thus Spake Zarathustra are available on the product page (https://ravens-eye.net/product/index.html), and more detailed examples can be found in the demonstrations pages (https://ravens-eye.net/applications/laboratories/demonstrations/index.html).  

Raven's Eye provides a number of ways to measure the similarities and differences between texts, including frequency and overrepresentation scores based on word use, grade-level scores, and context-based analytics.

tim

I'm using default tool that is build-into Total Commander (http://www.ghisler.com/). In case you are Window's user you can use simple console utility "fc.exe ".

I tried using jEdit with a Diff plugin, but that will give you the differences line by line like the solutions suggested above. I need something that will allow me to see if text has been displaced as well, or just reorganized. Word's document comparison comes close but it only recognizes deletion and insertion between two documents, and not 'moves'. I am actually even considering using our university's anti-plagiarism software to quantify percentage of similarity.

Maybe this will help you:

http://www.searchenginecaffe.com/2007/03/java-open-source-text-mining-and.html

You can use Total Commander file manager.

Select two text files with 'space' key.

Click on Files on the tool bar.

There should be Compare by content option.

You can compare in binary mode and in readable mode.

http://www.ghisler.com/

You could use a scoring tool that's used for speech recognition called Sclite. It compares two texts and reports the difference between them as word error rate (WER). It's really easy to use:

http://www1.icsi.berkeley.edu/Speech/docs/sctk-1.2/sclite.htm

WER is calculated based on the number of correct words, the number of deletions, insertions and substitutions. The evaluation is explained in Sclite.

You can use WordStat, Content Analysis and Text Mining Software, to compare similarities between multiple documents: http://provalisresearch.com/products/content-analysis-software/

 I was wondering what you ended up using to compare the similarity.  I wanted to do something similar by first using a percentage as to how similar two documents are.  I was thinking to use a plagiarism tool as well.  Could you provide feedback on which tool you ended up using?

I finally just highlighted texts for 'retained, removed, introduced, or reorganized on the first and second document and then counted manually. My texts were fortunately only a page or two and I only had pairs of texts, so it was manageable. I was hoping to find something that could work with longer and/or bigger quantities of texts.

Hello Kalliopi and others,

Raven's Eye (https://ravens-eye.net) automates the analysis of a number of types of textual natural language expressions, including transcribed survey responses and interviews, as well as written documents, books, and other texts.  Brief examples of it applied to U.S. political party platforms, and Nietzsche's Thus Spake Zarathustra are available on the product page (https://ravens-eye.net/product/index.html), and more detailed examples can be found in the demonstrations pages (https://ravens-eye.net/applications/laboratories/demonstrations/index.html).  

Raven's Eye provides a number of ways to measure the similarities and differences between texts, including frequency and overrepresentation scores based on word use, grade-level scores, and context-based analytics.

tim

I use Raven's Eye. The software has a function where you can upload and compare websites. It is really interesting. You can say choose 2 soft drinks. Access their websites and then use RE to analyze and compare the 2 websites of those sodas!

I agree with Tim- you can upload books and compare them. You can also upload social media sites and compare.

Another great thing is you can compare websites, texts and you can also upload audio and text files to analyze with Raven's Eye. This is a big plus for me!

For anyone doing content analysis or any other text analysis work -

your best bet to date is either:

1) Manual qualitative content analysis using NVivo, which also provides an 'autocode' function allowing to automatically apply your existing coding categories (that you are using for the comparison) to an imported body of text. This requires, however, at first an extensive manual coding process. If you search around the web, I'm sure you can find open source alternatives to NVivo.

2) You can automate most text mining, text analysis functions with the QDAP package in R. In combination with the RQDA, this should cover most your content analysis. I strongly recommend using R Studio, which makes the interface a lot more user-friendly. All the packages are open source and suitable for both qualitative and quantitative analyses.

Hello Kalliopi

I hope you are fine.

Would you have found an automatic solution since this post?

I am looking for exactly the same thing today.

Thank you

Sébastien

Total commander is nice solution for this issue. Download the tool for free from: https://www.ghisler.com/

Then follow procedure described by Mete Eray ^^

One way to solve this problem is using Jaccard index, we can tokenize and remove the no desired words first and after this we can calculate the Jaccard Index with the following structure:

Let A and B the texts to compare

J(A,B) = J(A ∩B)/ J(AUB) = J(A ∩ B)/ [J(A) + J(B) - J(A ∩ B)]

Where: 0

David Bustos Usta I have two different text files and want to compare them using Jaccard Index. Do you have a coding solution for that? I want to do it in Javascript but couldn't find any working code. Any help would be appreciated. Thanks