You can try this:

https://www.pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/

Probable solutions-

Assalamualikum sir, your problem is trivial. Underfitting occurs when your model doesn't learn properly, have less number of trainable parameters as well as the model complexity is less than required. So the first thing you need to do is an increase in the number of neurons in the dense layer. Like if you have a neuron size of 512x512 in the top, then try increasing it into 1024x1024 or 2048x2048 and so on. Then increase the number of epochs for training. More complex models mean more number of trainable parameters. For further contributions, try tuning hyperparameters of your model for better accuracy, recall, precision, f1-score and other evaluation measures. You can look upon this-

https://www.tensorflow.org/tutorials/keras/keras_tuner

You could try the following when your model underfits -

1. Try a different batch size

2. Try to gather more data

3. Augment data (if you havent)

4. Try EfficientNet or other modes, not just resnet and vgg

5. If you are restricted to vgg, change the number of layers present.

6. If its a known dateset, look at the general hyperparameters others have used.

7. Also verify your images, the models are pre-trained on imagenet and your data might be completely different, you could train the entire model again instead of transfer learning.

You can try this:

https://www.pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/

The attached image shows that the model suffers from overfitting, which means that the model performs better on training samples than on test samples. You can address this by using a regularization technique such as the dropout and early stopping techniques. You can check the links below for research papers that will give you more information.

Article An optimized deep learning architecture for the diagnosis of...

Article An optimized model based on convolutional neural networks an...

You can try this:

https://www.pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/

If the response is helpful then click on Recommend to help others.Thank you.