Hello members,
I am about to write my proposal and I decided to work on the Attention mechanism in AI.
Does anyone have experience in this field?
which area of artificial intelligence not covered yet by the attention mechanism?
I need to know what is the latest work done in this field? Any problem facing it? suggesting some papers that could help me to explore it more?
Thanks in advance,
Dear Ali El Romeh the Attention mechanism in Artificial Intelligence use in different areas. It is the detection of objects on the video and pictures, a selection of voice commands, a search of words in the texts etc. As areas of artificial intelligence which not enough covered yet by the attention mechanism, I'd like to highlight the multidimensional attention, the detection and the identification of multiple different objects in the video, voice etc.
Ali El Romeh ,
Attention is a very broad field and you have to be more specific. For example, the attention mechanism can be done at the level of vision [1]. This was integrated in the Kismet robot with a habituation routine [2]. Implementing this for audio would require a different strategy such as source isolation and energy computation.
Another view on attention is because of relevance based on previous information. This would require meta learning [3] or learning to learn what is relevant and discard the rest in neural networks.
I would suggest that you narrow your field of research towards a specific goal
Regards
[1] Scassellati, B. (1998). Finding eyes and faces with a foveated vision system. In Proceedings of the American Association of Artificial Intelligence (AAAI-98) (Vol. 86).
[2] Breazeal, C. L. (2002). Designing sociable robots. MIT press.
[3] Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020). Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439.
You can read this to have an idea about attention mechanism in deep learning
https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.analyticsvidhya.com/blog/2019/11/comprehensive-guide-attention-mechanism-deep-learning/&ved=2ahUKEwi02fed27nxAhWewjgGHbo-AVIQFjADegQIGBAC&usg=AOvVaw1dtsepF__xKhaR20akyx6x
Also you can traverse attention in Natural Language Processing from here
https://www.google.com/url?sa=t&source=web&rct=j&url=https://arxiv.org/pdf/1902.02181&ved=2ahUKEwi02fed27nxAhWewjgGHbo-AVIQFjACegQIBhAG&usg=AOvVaw3yQ3VxWqW03Ua0gfCYJ1Sm
Dear Ali El Romeh
I recommend to read these articles first:
1. https://towardsdatascience.com/intuitive-understanding-of-attention-mechanism-in-deep-learning-6c9482aecf4f
2. https://www.analyticsvidhya.com/blog/2019/11/comprehensive-guide-attention-mechanism-deep-learning/
Kind Regards
Qamar Ul Islam
There are hundreds of thousand AI problems to be solved and need millions of new researchers. PLEASE Start a Literature review over the topic of your interest that may be preferably based on your previous academic background. Try to understand what has been done in your area of interest and what is need to be contribute in it? Highlight this gape/research question. Then think about how it will benefit the world? how you would like to do it? It will need some more literature review with addition of your own abilities to solve the problem, compile it and proceed further accordingly as the requirement by your organization.
Dear Ali El Romeh
There is a recently published review paper on use of attention mechanism for deep learning-based remote sensing image processing that may help you to identify the gaps and plan your future work. Have a look at the following paper:
Article Effect of Attention Mechanism in Deep Learning-Based Remote ...
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence