Hello ResearchGate community
I am working on a deep learning (CNN + AEs) approach on facial images.
I have an input layer of 112*112*3 of facial images, 3 convolution + max pooling + ReLU and also 2 layers of fully connected with 512 neurons with 50% dropout to avoid overfitting and last output layer with 10 neurons since I have 10 classes.
I have also used reduce mean of softmax cross entropy and also L2.
For training I divided my dataset to 3 groups of:
60% for training
20% for validation
20% for evaluation
The problem is after few epochs the validation error rate stay fixed value and never changes. I have used tensorflow to implement my project.
I hadn't such problem before with CNNs so I think it's first time. I have cheked the code it's based on tensorflow documentation so I don't think if the problem is with the code. maybe I need to change some parameters but I am not sure.
Any idea about common solutions for such problem?
Can you retry the splitting into train -validation-test (actually try doing the split a few different times and training the network)
with splitting only once, there is a slim chance that your validation and training are different in terms of the features captured by the network.
also, does the training error decrease.
First thanks for your answer. Actually the dataset has been spillited randomly every time I run the training program. it is shuffled for 5 times then I split it into 3 part as I mentioned earlier.
One thing I noticed just now is that only few images in my dataset are whole black 50/6676. I want to eliminate them and try again. what's your opinion?
Could you give more details about your CNN architecture?
Did you define the architecture by yourself or did you use an architecture from another work?
Your training process seems OK to me. Maybe the problem is in your architecture, since your problem is not converging.
I guess that your network is learning nothing. CNNs generally require a large amount of training data. Since you say that you have only ten classes, I assume that your training set is simply not large enough. With little training data, the network will over-adapt to the training instances - a simple dropout layer might reduce this effect, but not much.
Another factor might be the learning rate. My experience is that too large learning rates result in the opposite effect: the network always outputs the same value, independent of the actual input. Thus might also explain your outcome.
It is typical in situations where the NN is not possible to improve. In such cases, it is recomendable to increase the number of hidden neurons
- Badges
- Science topic