Hey Muhamamd,

depending on what data you are using there might be a pretrained GAN. With that you could start generating new images which then can be preprocessed to suit your model specifications. I don't think that you should generate them at runtime. Imagine having batches of 128 images that you want to pass to your model for one training iteration. You would have to start your GAN, extract and preprocess the image and pass it to the batch. The generated images would then be passed to train your model. (ideally a batch would have a large portion of original data and small portion of generated ones to not skew your metrics).

It would be much easier to just generate some images with a GAN first before training your actual model. And then to mix them with your original data. Maybe you should also keep in mind that the ratio of original and generated images should be carefully chosen. My suggestion would be to start with original (100%) + 5% generated images and then maybe create a second model which is trained with a larger portion (lets say +10%, +15%, +20%) of generated images on top of the original data set to compare performance between them. Generated images might impair your model performance, which is why you should keep track of the portions you add.

If you don't want to persist the images on the disk the best way imo would be to let each model run as long as possible. Let the GAN generate as much as possible in one go for each iteration by choosing a larger batch size.

Yes, GAN can be used to "hallucinate" additional data as a form of data augmentation.

See these papers which do pretty much what you are asking:

https://arxiv.org/abs/1711.04340

https://arxiv.org/abs/1801.05401

https://arxiv.org/abs/1803.01229

If your GAN is sufficiently well trained, there's no reason why this shouldn't help improve model performance. If your GAN is bad, you'll get garbage.