Please find attached the picture. I have a bigger image (in orange) with two sub-sections (in blue). I have to translate (superimpose) different regions within these sub-sections with another small image (in black). I have a collection of these small images and bigger images and would like to do this image translation (at the image level and not style transfer that happens at the feature map level) using deep learning. Kindly suggest the best algorithm and strategy that can be used for this purpose.
If I understand you question correctly the following might work:
1) Feed the large image into a pre-trained CNN (withouth the dense layers) using the full resolution. You'll get a feature tensor of height x width x features
2) Feed the small image into the same pre-trained CNN (also at its original resolution). You'll get a feature tensor of height_small x width_small x features
3) Convolve the small feature tensor with the large feature tensor.
4) Find the maximum overlap
Benjamin Staar
Although the question was not clear, your answer maybe is what he is looking for. However, don't you think that pre-trained CNN models normally require fixed dimensions to work? If so, then am not sure how to input the said images at different resolutions?
Since the convolutional layers have spatially fixed connections you could theoretically process arbitrarily large input resolutions. There is only a lower bound due to the pooling operations. In principle you could even include the dense layers by fixing their spatial connectivity (effectively turning them into conv layers)
Well, the fact that existing pre-trained model architectures require input images with fixed size, could be an issue, what's your take on that?
That's only true if you include the dense layers at the end, since their spatial connectivity is not fixed. In tensorflow/keras you might have to make sure that all input images have the same input resolution though. Pytorch even seems to work with differently sized input images (as long as they are not too small).
...then during the convolution, one would stumble against the size of the filters (kernel size) , the size of the output feature map, etc. That said, I can see the advantage of standardizing the size of the input images.
I am not sure I follow.
You could view your pre-trained conv net as a very complex filter, that you convolve your (padded) input image with. Due to pooling, the dimensions of your output feature map are smaller, which is why there is a lower boundary to the input size: you can not apply 2x2 strides/pooling or 3x3 filters if the inputs spatial dimensions are too small (is that what you mean?)
- Badges
- Science topic
- Similar topics
- Computer Science
- Images