Hi,
I want to apply GAN as a data-augmentation technique in my model. I am bit confused regarding implementation that how could I give the generated images of GAN to my model on run-time? because GAN takes time to generate good and sensible images. At what point I could feed GAN generated Images to my model and what would be the best approach? In whole scenario, I want to skip saving Images on the disk and then preprocessing the images to feed into my model.
Looking forward to your kind suggestions!
Hey Muhamamd,
depending on what data you are using there might be a pretrained GAN. With that you could start generating new images which then can be preprocessed to suit your model specifications. I don't think that you should generate them at runtime. Imagine having batches of 128 images that you want to pass to your model for one training iteration. You would have to start your GAN, extract and preprocess the image and pass it to the batch. The generated images would then be passed to train your model. (ideally a batch would have a large portion of original data and small portion of generated ones to not skew your metrics).
It would be much easier to just generate some images with a GAN first before training your actual model. And then to mix them with your original data. Maybe you should also keep in mind that the ratio of original and generated images should be carefully chosen. My suggestion would be to start with original (100%) + 5% generated images and then maybe create a second model which is trained with a larger portion (lets say +10%, +15%, +20%) of generated images on top of the original data set to compare performance between them. Generated images might impair your model performance, which is why you should keep track of the portions you add.
If you don't want to persist the images on the disk the best way imo would be to let each model run as long as possible. Let the GAN generate as much as possible in one go for each iteration by choosing a larger batch size.
Yes, GAN can be used to "hallucinate" additional data as a form of data augmentation.
See these papers which do pretty much what you are asking:
https://arxiv.org/abs/1711.04340
https://arxiv.org/abs/1801.05401
https://arxiv.org/abs/1803.01229
If your GAN is sufficiently well trained, there's no reason why this shouldn't help improve model performance. If your GAN is bad, you'll get garbage.
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering
- Signal Processing
- Image Processing