I have hyperspectral images of size (12x12x11) where 12x12 is the spatial dimension and 11 is the depth of the images.. the corresponding output labels are images of 12x12x91 dimension and i want to train them using a decoder sort of a network?
my pixel values are between 0 and 1.
what output layer activation should i use and what loss function should i keep ?
Yes......
With the main difference that shallow attention fusion only summarize attention information at the last time step of the decoder (obviously the wrong way), and deep fusion summarizes attention information at each step, and merge hidden state of the encoder into the decoder at each time step, and such merging process is a weighted sum based on how much attention given decoder time step has to each time step from the encoder.
The encoder decoder architecture started the recent neural machine translation trend. I'd refer to it as old school if it were more than a few years old. Shockingly, as the name implies, there are two components - an encoder and a decoder.
For a word level encoder-decoder machine translation system, such as the depiction below:
- Take an recurrent neural network (RNN) - usually an LSTM - and encode a sentence written in language A (English).
- The RNN spits out a hidden state, which we refer to as S.
- This hidden state hopefully represents all the content of the sentence.
- This hidden state S is then supplied to the decoder, which generates the sentence in language B (German) word by word.
The encoder decoder showcased the potential that neural based machine translation may provide. Even with modern complex neural machine translation architectures, the majority of them can still be decomposed in terms of the encoder-decoder architecture.
Can you provide a full description of the dataset, including what sort of labels (if any) you have for the images? Also, briefly explaining the significance of the input images as well as the output images will help.
Shounak Datta i basically have hyper spectral images of dimensions 12x12x91.
The whole image represents a spice mixture. Different images represent different spice mixtures.
Each image has a specific mixture ratio so all 12x12 pixels will have the same groundtruth values which i feel lead to redundancy.
the corresponding groundtruth are images of 12x12x11 dimension where the 11 dimension basically represents a vector consisting of the ratio of the mixtures of spices in each pixel. for eg the 11 dimension vector has values that sum to 1 and have values such as [0.5,0,0,0,0.5,0,0,0,0,0,0],[0,0,0,0.25,0.5,0,0,0,0,0,0.25].
The image pixel values are between 0 and 1.
I want to create a decoder and use 3d convolution and upsampling to use the ground truth as the input so a 12x12x11 input and a 12x12x91 output to try to predict the corresponding image for each ground truth input image
Dear Muneeb,
I like your idea. I think it is quite realistic and can be accomplished. You can use a GAN sort of architecture to do this. Have a generator map from the 12x12x11 input space to a 12x12x91 output space. Also have a discriminator map from the 12x12x91 space back to the 12x12x11 space. Then do as follows:
1. Train the discriminator on the labelled data to output the correct 12x12x11 images given the 12x12x91 input images.
2. Use the generator to generate some fake 12x12x91 images for some target 12x12x11 images.
3. Train the discriminator to output all zeros of size 12x12x11 for the generated fake images.
4. Train the generator to generate 12x12x91 images that can fool the discriminator into outputting the correct 12x12x11 target images used to generate the corresponding images.
This is only a preliminary plan, you'll need to make changes. Good luck!
- Badges
- Science topic