I have studied about attention mechanism and have seen its application in Natural Language Processing (NLP) and Computer Vision(CV). NLP is not my area of interest. However, CV is my area of interest but attention mechanism is found to be applicable in Image Captioning, which is a part of Information Retrieval. Image Captioning deals with long subsequences of text to convert unstructured image-related data to structured data. It does not relate directly with images but extracts text-based information from images. I want to use attention mechanism directly upon images, through its integration with CNNs especially for better feature extraction , selection and classification. Suggestions would be of great help.
Dear Dolly Das,
To my understanding the multi head attention mechanism is basically splits the input vector of the neural network into sub-vectors. If we imagine the input as a high dimensional space, the attention mechanism is dealing with lower level projections. Meaning it can be used for image processing as well. I think it depends on how the image is transformed into input vectors. I have no experience in that. In NLP we use for instance word embedding mechanism to this purpose (e.g.: word2Vec).
Kind regards,
László
Dear Dolly Das
These papers might help, have a look:
1. https://mdpi-res.com/d_attachment/remotesensing/remotesensing-13-02965/article_deploy/remotesensing-13-02965-v2.pdf
2. https://mdpi-res.com/d_attachment/electronics/electronics-09-02022/article_deploy/electronics-09-02022-v2.pdf
3. https://arxiv.org/pdf/1811.08513.pdf
4. https://www.hindawi.com/journals/omcl/2021/6280690/
5. Chapter Feature Extraction by Using Attention Mechanism in Text Classification
6. https://iopscience.iop.org/article/10.1088/1742-6596/1693/1/012173/pdf
Kind Regards
Qamar Ul Islam
Dear Dolly Das ,
You can find a comprehensive explanation of developed attention mechanisms for remote sensing image processing (also can be employed for other image processing applications) and how to add them to your deep neural network using the following paper:
Article Effect of Attention Mechanism in Deep Learning-Based Remote ...
- Badges
- Science topic
- Similar topics
- Correction