Hi ,
I know that most of existing probabilistic and statistical term-weighting schemes (TF-IDF and its variation) are based on linked independence assumption between index terms. On the other hand, semantic information retrieval are seeks the importance of linked dependence between index terms each other.
Please, I am wondering when linked dependence between index terms is vital ? When also can we neglect linked dependence between index terms?
Note: dependence assumption: if two index terms have the same occurrences in the document, this will tend to that index terms are dependent and they should have the same term-weight values.
Thanks
Osman
Taking into account the dependency or not between indexs (terms) depends on the objective of your research. For example, for classification problems the interrelation is not very interesting, because we what assign a class to a document.
On the other hand for information retrieval problems, it matters a lot, especially if you are doing semantic searches. Another situation to take into account is in the creation of summarization of texts.
I hope it helps. Regards
Hi Vladimir,
Thank you for your answer, but in Information Retrieval, the partially judged document collections have an issue with relevance judgement values. Thus, I think, term- weights should have partially semantic relation such as term-weights dependence in unjudged documents. However, the text classification problem has not this issue.
https://link.springer.com/chapter/10.1007/978-3-319-27060-9_7
Best wishes,
Osman
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining