I have certain concepts that I want to match in longer sentences. I have about 50 concepts like this:
chief accepted policy,
chief rejected policy,
proposal was deferred,
...
proposal accepted consensus,
no consensus proposal rejected,
I am trying to draw up a diagram of how this organisation makes decsions regarding proposals. A proposal comes in every week. I have text files of all their email messages where they state these actions. I break the messages to paragraphs by identifying blank lines and then by using standforn NLP to break into sentences. Unfortunately their email messages contain long sentences.
I have tried sentence similarity using ws4j (wordnet), text classification (weka), string matching to match these concepts to sentences (it would be difficult to paste code here). But these does not give the desired result. I am going to look at word order similarity but I dont thing it would help. Bag of words, Phrase matching also wont help.
Here is why its not giving result. Its mainly because the sentence contains so many other things. Trying string matching with these words in sentences “chief” “accepted” “proposal”. Here I am looking for instances when chief actually accepted the proposal.
Meaningful capture:
"the revised proposal was accepted by chief at seminar."
Inaccurate capture: But also inaccurate captures happen which don't mean the same thing : same words also exist in these sentences and its not same meaning.
"Chief, i 'm particularly interested in hearing your opinion about the following three issues : should this proposal be accepted at all . ."
"Chief said, i will retract the proposal, because is too late for me.is the proposal accepted ? ."
"hi chief, should this proposal be accepted at all . ."
Opposite meaning Also sometimes the sentences contain the opposite meaning like: "Chief has decided that this proposal will not be accepted"
Anyone has any idea, please advice how to go about doing this.
I want to extend this study to more organisations after this organisation. They may use other methods of dealing with proposals. Therefore, I believe string matching, bag of words, regex matching might not work.
You need to build yourself a semantic tag set that represents all the different concepts you are trying to recognise. Then you need to annotate a corpus of documents with your tag set so that you have a gold standard. Then you need to use an SVM package or similar classifier to build a model of your tag set. Arriving at a suitable tag set will require some experimentation with what semantic concepts will be useful and identifiable computationally. For example in your examples above you have to decide if you want tags for participants, decision verbs, and decision objects as distinct to tags for phrases that represent those three objects as one. I suggest the former approach is likely to be better.
Hi,
As per your explanation, your data-set contains lots of objective sentences as compared to subjective sentences. First of all you have to classify these sentences other wise the probability of getting wrong results is very high. Only subjective sentences are of your concern. Once you have done with this, you can use some aspect extraction approaches by using your concept corpus. After you have the required concepts, then with the help of sentence structure, you can identify whether it contains opposite meaning or not.
For detecting subjectivity, you can look into following book.
https://www.cs.uic.edu/~liub/FBS/NLP-handbook-sentiment-analysis.pdf
and for aspect extraction approaches, which suits you, you can look into following paper,
http://link.springer.com/article/10.1007/s10462-016-9472-z
Hope this will help.
- Badges
- Science topic