Is there a (preferably open-source) tool available that generates co-occurrence tables for n-grams? I.e.: that can tell you which n-grams a bigram like "water security" tends to co-occur with within an certain (user-defined) 'window' - say within 2 sentences before or after its occurrence in a sentence?
You can try the Ngram Statistics Package by Ted Pedersen'.
http://www.d.umn.edu/~tpederse/nsp.html
While WebCorpus open source tool http://sourceforge.net/projects/webcorpus/ does not do this out of the box, it already computes n-gram statistics and co-occurrence counts in a scalable way, i.e. you could use this on web crawls or any other coprora.
The adaptation to n-gram co-occurrence as you describe it should be quite straight-forward, if you have some command of Java. Feel free to get in touch if you have questions!
WordStat could be helpful: http://provalisresearch.com/products/content-analysis-software/
Thank you for your posts. Actually, I need to train parallel aligned en-fr corpus, aligned at the level of sentences found at http://opus.nlpl.eu/EMEA.php. Here I need to ask you whether is possible to use wordstat and if is it capable of sorting out n-grams test corpus from these EMEA corpus.
EX.,
Recurrence prevention of manic episodes in Bipolar I Disorder: Prévention des récidives des épisodes maniaques dans les troubles bipolaires de type I:
Somewhere I am confused to extract key phrases or n-grams of medical use as severe manic episodes, lung cancer, etc , this is a medical aligned corpus.
I will use it for neural machine translation task, in other words, I need to construct a test set to test and measure the output precision trained by tensorFlow .
THanks in advance
If I get your question, I think you could try ANTCONC https://www.laurenceanthony.net/software/antconc/
or TXM https://txm.gitpages.huma-num.fr/textometrie/
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python