Hello,
I'm trying to train a model to learn from some images and my target is some images too.
I want to give RGB images to my model and get the D channel (Depth) as output. how can get a 2D image as the output of the CNN neural network? is this the best way for my task? can I use GAN or other networks for this task?
yes, you can use autoencoder and GANs for learning image to image (as input and output). @mohammad hasani
Method 1: Use autoencoder.
Make architecture with input (I) X*Y*3 and reduce gradually to latent(encoder) representation of size say 512x1. Then upscale it back by the same number of layer dimensions. However, at the last layer, make it X*Y*1 instead of X*Y*3. Use the sigmoid layer to get the predicted output (P). Then, optimize the following MSE (summed over all training images):
1/2(P-D)^2, where P is output and D is the ground truth depth map.
Method 2: Use VAE ( variational auto encoder)
Similar to method 1 with MSE function. However, it has added constraint on making divergence smaller between the distribution of ground truth (D) vs predicted (P).
Method 3: Use GAN.
There is a lot of literature that talks about depth estimation using RGB images and GANs. For ex:
https://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w21/Lore_Generative_Adversarial_Networks_CVPR_2018_paper.pdf
As answered before, yes, a CNN is a structure suited for this work.
Strictly speaking, if you're not reconstructing your input, it is not an autoencoder (as far as I know) but a conventional neural network - but that's a detail :)
A GAN in this case would be applicable, but I don't see the benefit. I would recommend starting with a traditional CNN - given the case you have labeled depth images.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training