Hi I used the below program for fine tuning VGG16 for 5 classes (11600 for each class) following the example in https://keras.io/applications/ . It gives a good validation accuracy, which is around 93%. However, I used the same architecture to solve 2 classes selected from the same 5 classes (binary classifier), the validation accuracy always stays around 42%. I expected to get better results. Is the problem from the model or the data? Any advice?
Thanks