U-net Neural Networks are similar to GAN and consist of a contracting path and an expansive path.
In this regard can we consider that U-Net a part of Generative Adversarial Networks (GAN) Architecture? Can be GANs use for Segmentation pictures similar to U-Net?
Thank you, dear Omar Ibrahim Obaid , that you confirmed my assumption.
Your link is very interesting in this regard.
U-Net discriminators focus more on semantic and structural changes between real and fake whereas, in GAN two neural networks compete with each other to become more accurate in their predictions by creating their own training data and automatically discovering and learning regularities to generate new samples that plausibly could have been drawn from the original dataset by framing the problem as a supervised learning problem with two sub-models. The generator generates new examples and the discriminator tries to classify these examples as either real (from the domain) or fake (generated).
Thank you, dear Imtiaz Husain . Your update is very important. As I understound the significant difference GAN architecture from U-Net is use in GAN two neural networks. But these two neural networks can be U-Nets. In my opinion, in the general case, GAN can include multiple discriminators networks not only two.
- From an architectural perspective, U-Net is a CNN followed by a symmetric Deconvolutional network. While the first part (i.e. the CNN part) does the usual convolution to discover the important features, the deconvolutional part does an up-sampling. This is quite different from GANs where two networks compete with each other. Please look at the original publication of U-Net for more details at the following link:
Article U-Net: Convolutional Networks for Biomedical Image Segmentation
Thank you Sudip Sanyal , yes, your approach to this question is more strong.
Yes, if you are using Pix2Pix GAN then U-Net is definitely a part of GAN (in the generator model). Please check the mentioned link:
https://github.com/ayanbabusona/Deep_Learning_with_Python/blob/main/GAN/Pix2Pix_GAN.ipynb
Basically GAN has two network. One is the discriminator which will try to discriminate between real and fake images. And the second network of the GAN is the generator which will be responsible to generating fake images but very close to the original images rather we can say that the fakes images will be in the same distribution of the original images.
UNET basically used in image segmentation problems where the input and output dimensions are same.
GAN tries to generate new images from a random noise. So the generator of the gan have very few dimension in input but output is in large dimension.
You can use UNET as the generator in the GAN ( pix2pix uses this) .In this case , you will generate fake images not from random noise but from other reference image. You have other choices like deconvolution network, auto encoder etc. It will truly depend on your problem.
From training perspective GANs are “training methods (configurations)” for emphasizing “creativity of generative architecture” (e.g. CNN) where U-Net can be seen as generative CNN. [https://arxiv.org/abs/1406.2661, https://arxiv.org/abs/1505.04597]
Question 1: yes in this aspect U-Net can be used as a part of GANs.
Question 2: yes, hire GAN has a segmentation network S (or generative G depending on configuration) and a discriminator D. For GAN and U-Net details, s. U-GAN [https://ieeexplore.ieee.org/document/8845397].
Other examples are (1) https://arxiv.org/abs/2104.09917 designed an efficient GAN network for image semantic segmentation, (2) SegGAN https://ieeexplore.ieee.org/document/8499105, (3) GAN Mask-RCNN https://arxiv.org/abs/2010.13757and etc.
- Badges
- Science topic