Maybe you can specify the task a little more in detail.
There are several options that may be adressed with the chosen key words.
I once used dot-plots as a pretty powerfull and visual tool.: http://code10.info/index.php?option=com_content&view=category&id=52:cat_coding_algorithms_dot-plots&Itemid=76&layout=default
Jan Schulz, I try to find out the similarity score between two documents or articles. How semantically related each other?
How many characters are the typical texts you want to compare?
When text sequences are short it might be an option to use an edit distance, like e.g. Levenshtein:
https://en.wikipedia.org/wiki/Levenshtein_distance
For more complex texts I would give the dot-plots a try. You simply see similarities as bright structures along the diagonal (or with an offset, when texts have sematic similarities but being shifted in relation to their position in the text or change in order of words. The result depends a bit on the window size. A very simple (and no longer up-to-date) try could be this: http://www.code10.info/index.php?view=article&catid=50%3Acat_coding_software_serolis&id=63%3Aserolis-software-package-for-dot-plot-creation&option=com_content&Itemid=61
Simply create one or two empty new sequences, double click, paste your text and choose "dotplot creation" with option "literal".
Nevertheless you may have a look into Dan Gusfield's "Algorithms on strings, trees and sequences"...
First, you should use SBERt to create embeddings of the sentences. Then, utilize an algorithm like UMAP to reduce dimensionality. If you want to highlight latent topics, use HDBSCAN for high-density clustering. Finally, use the TensorFlow Projector to check for sentences similar in cosine similarity. I explain this in this publication: https://www.linkedin.com/feed/update/urn:li:ugcPost:7076949962558705664?commentUrn=urn%3Ali%3Acomment%3A%28ugcPost%3A7076949962558705664%2C7077287447948070912%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287077287447948070912%2Curn%3Ali%3AugcPost%3A7076949962558705664%29
- Badges
- Science topic
- Similar topics
- Agricultural Science
- Agronomy