Dear Semeh Ben Salem

I have been working with NLP for quite some time. There would be a lot to share over here, but it would be very difficult. I would however like to share a few points about my experience, especially for researchers who are just starting out or would like to work in this field. Many areas of NLP can be explored, such as text summarization, text classification, named entity recognition, relationship extraction, etc. ML is an essential part of NLP, but there is also a need for linguistic rules and dictionaries. Often the preprocessing and cleaning of text data makes working with it extremely difficult. NLP is still in its growing phase and one needs to be aware of what they can expect as result. Several good blogs and articles are available, such as the following. Thanks

https://www.analyticsvidhya.com/blog/2021/04/role-of-machine-learning-in-natural-language-processing/

https://monkeylearn.com/blog/nlp-ai/

https://towardsdatascience.com/natural-language-processing-nlp-for-machine-learning-d44498845d5b

https://www.lexalytics.com/lexablog/machine-learning-natural-language-processing

https://medium.com/fair-bytes/nlp-bias-against-people-with-disabilities-bf13bda66a8e

https://www.nap.edu/read/2308/chapter/23

https://insidebigdata.com/2021/01/03/3-nlp-trends-prime-for-improvement-in-the-new-year/

https://heartbeat.fritz.ai/the-7-nlp-techniques-that-will-change-how-you-communicate-in-the-future-part-i-f0114b2f0497

https://towardsdatascience.com/nlp-based-data-preprocessing-method-to-improve-prediction-model-accuracy-30b408a1865f

https://medium.datadriveninvestor.com/challenging-and-improving-nlp-model-in-proper-manner-ec2ed2d7629d

https://www.kdnuggets.com/2019/01/solve-90-nlp-problems-step-by-step-guide.html

Dear Semeh Ben Salem

Basically, NLP is a field in machine learning with the ability of a computer to understand, analyze, manipulate, and potentially generate human language. Information Retrieval(Google finds relevant and similar results). Information Extraction(Gmail structures events from emails).

Through combining syntactic and semantic techniques for analyzing text, computers can gain access to a deeper understanding of spoken words. Syntactic describes the grammatical structure of a sentence and semantic the meaning conveyed by it. This is how a computer is able to process natural language.

Kind Regards

Qamar Ul Islam

Thanks for all of you for your answers, it would also be very appreciated if you could share with us some issues and difficulties faced while working on a NLP project and how to solved it. It can help others who are facing similar issues.

References to the works of prof. E. V. Lutsenko on ASС-analysis of texts

ASС-analysis of texts allows you to:

- to form generalized linguistic images of classes (semantic cores) based on fragments or examples of texts related to them in any language;

- quantitatively compare the linguistic image of a particular person, or the description of an object, process with generalized linguistic images of groups (classes);

- compare generalized linguistic images of classes with each other and create their clusters and constructs;

- to investigate the modeled subject area by studying its linguistic system-cognitive model;

- to carry out intellectual attribution of texts, i.e. to determine the probable authorship of anonymous and pseudonymous texts, dating, genre and semantic orientation of the content of texts;

- all this can be done for any natural or artificial language or coding system.

Lutsenko E. V. Synthesis of semantic cores of scientific specialties of the Higher Attestation Commission of the Russian Federation and automatic classification of articles by scientific specialties using ASС-analysis and the intellectual system " Eidos "(on the example of the Scientific journal KubGAU and its scientific specialties: mechanization, agronomy and veterinary medicine) / E. V. Lutsenko, N. V. Andrafanova, N. V. Potapova / / Polythematic network electronic scientific Journal of the Kuban State Agrarian University (Scientific Journal of KubGAU) [Electronic resource]. - Krasnodar: KubGAU, 2019. – №01(145). P. 31-102. - IDA [article ID]: 1451901033. - Access mode: http://ej.kubagro.ru/2019/01/pdf/33.pdf, 4,5 cu. p. l.

Lutsenko E. V. Formation of the semantic core of veterinary medicine by automated system-cognitive analysis of passports of scientific specialties of the Higher Attestation Commission of the Russian Federation and automatic classification of texts in the areas of science / E. V. Lutsenko / / Polythematic network electronic scientific Journal of the Kuban State Agrarian University (Scientific Journal of KubGAU) [Electronic resource]. - Krasnodar: KubGAU, 2018. – №10(144). P. 44-102. - IDA [article ID]: 1441810033 – - Access mode: http://ej.kubagro.ru/2018/10/pdf/33.pdf, 3,688 cu. p. l.

Lutsenko E. V. Intellectual binding of incorrect references to literary sources in bibliographic databases using ASС-analysis and the Eidos system (on the example of the Russian Science Citation Index-RSCI) / E. V. Lutsenko, V. A. Glukhov / / Polythematic network electronic Scientific Journal of the Kuban State Agrarian University (Scientific Journal of KubGAU) [Electronic resource]. - Krasnodar: KubGAU, 2017. – №01(125). P. 1-65. - IDA [article ID]: 1251701001 – - Access mode: http://ej.kubagro.ru/2017/01/pdf/01.pdf, 4,062 cu. p. l.

Lutsenko E. V. Application of ASС-analysis and the intellectual system "Eidos" for solving in general the problem of identifying literary sources and authors according to standard, non-standard and incorrect bibliographic descriptions / E. V. Lutsenko / / Polythematic network electronic scientific Journal of the Kuban State Agrarian University (Scientific Journal of KubGAU) [Electronic resource]. - Krasnodar: KubGAU, 2014. – №09(103). P. 498-544. - IDA [article ID]: 1031409032 – - Access mode: http://ej.kubagro.ru/2014/09/pdf/32.pdf, 2,938 cu. p. l.

Lutsenko E. V. ASС - analysis of the problems of articles of the KubGAU Scientific Journal in dynamics / E. V. Lutsenko, V. I. Loiko / / Polythematic network electronic scientific Journal of the Kuban State Agrarian University (KubGAU Scientific Journal) [Electronic resource]. - Krasnodar: KubGAU, 2014. – №06(100). P. 109-145. - IDA [article ID]: 1001406007 – - Access mode: http://ej.kubagro.ru/2014/06/pdf/07.pdf, 2,312 u. p. l.

Lutsenko E. V. Attribution of anonymous and pseudonymous texts in system-cognitive analysis / E. V. Lutsenko / / Polythematic network electronic scientific Journal of Kuban State Agrarian University (Scientific Journal of KubGAU) [Electronic resource]. - Krasnodar: KubGAU, 2004. – №03(005). P. 44-64. - IDA [article ID]: 0050403003 – - Access mode: http://ej.kubagro.ru/2004/03/pdf/03.pdf, 1,312 y. p. l.

Lutsenko E. V. Attribution of texts as a generalized problem of identification and forecasting / E. V. Lutsenko / / Polythematic network electronic scientific Journal of the Kuban State Agrarian University (Scientific Journal of KubGAU) [Electronic resource]. - Krasnodar: KubGAU, 2003. – №02(002). Pp. 146-164. - IDA [article ID]: 0020302013. - Access mode: http://ej.kubagro.ru/2003/02/pdf/13.pdf, 1,188 cu. p. l.

D Lutsenko. S., E Lutsenko.V. Intellectual dating of the text, determination of authorship and genre on the example of Russian literature of the XIX and XX centuries, 2020 // The article is in an open archive. 38 p. – AS DOI: 10.13140/WG.2.2.28824.01281, https://www.elibrary.ru/item.asp?id=43796415

Lutsenko D. S., Lutsenko E. V. Intellectual attribution of literary texts (finding the dates of the text, determining authorship and genre on the example of Russian literature of the XIX and XX centuries), 2020 // Article in the open archive. 9 p – - DOI: 10.13140/RG. 2.2.15349.81122, https://www.elibrary.ru/item.asp?id=43794562

Since natural language is context dependent, Natural Language Understanding should be performed in a context-dependent manner, rather than attempting to process it like a programming language.

The Computational model laid down by Alan Turing in the Universal Machine is one of Attribution, where a name (as an E-square) indicates that a data (F-square) is to be marked, and this name-value model was passed into the computing world, in terms of memory addresses (represented by a name in source code) referencing a data value (byte or bytes) This name-value model can also be seen in filenames and data or URLs and web pages, and popular 'digital assistants' use, for example, JavaScript or annotated Java to do this processing.

The Linguistic model of Denotation, on the other hand, uses data to reference data (where each data value is an utterance), so if I say to you "if we are holding hands, whose hand am I holding" you can imagine the idea, and make a reply. We might call these data items an utterance and reply, but these names play no part in the understanding. The action and values in the traditional model are intrinsic within the data.

The Linguistic notion of Connotation - what something implies - can be used to build this process, for example, "I need coffee implies that you add coffee to my needs list" Thus we can construct algorithms by voice - at the point of use - rather than depending on a programmer's understanding of what you want

Because an utterance can be reduced to a very large integer, all we are simply swapping (very large) numbers - numbers that can be generated by speech-to-text software.

I do hope this makes some sense,

Martin

P.S. This thesis is evidenced in software:

https://github.com/martinwheatman/enguage

Deleted research item The research item mentioned here has been deleted

For example, like this: http://lc.kubagro.ru/aidos/Works_on_ASK-analysis_of_texts.htm

dear Eugene Veniaminovich Lutsenko thank you very much for your answer. However, could you please provide us with an English link.

https://www.researchgate.net/publication/343214991

NLP application in the context of social media and COVID-19. To extract people's reactions to the reopening policies.

Article Anatomy into the battle of supporting or opposing reopening ...