I have designed a CNN from scratch to identify certain anomalies in EEG. My input to the CNN is the EEG data (16 channels and 64 samples per epoch). The CNN has 2 convolutional layers. Each layers has a 1-D filter of size 1x32 and 1x8 respectively. The number of filters in each layers is 32 followed by 64. When I train the CNN for 100 epochs, even though the training accuracy reaches 90% or more, the training loss saturates at 0.5 .
Is there a reason for this? Is there any way to bring down the loss?
Thanks for your time and help.
Hi,
What kind of loss function are you using ? (MSE criteria maybe ?) And, what is the validation loss ? is it much higher than the training loss ? if that's the case, then you are indeed overfitting the network. One way to assuage it is to use a pooling layer (the conventional convolutional layer is: Conv->ReLU->Pooling). And, a fully connected layer after the ConvLayers might help.
Hi Shahin,
The loss function in MATLAB by default is 'cross-entropy'. Yes, I have used ReLU and Maxpooling layers after my convolution layers.
inp layer, [conv1->ReLu->Maxpool]->[conv2->ReLu->Maxpool]->f.connected->S.Max->Classification Layer.
This is my overall CNN structure.
As for the validation loss, it is less than the training loss but not much. I have also tried varying the learning rate but still no effect on the training loss.
" And, a fully connected layer after the ConvLayers might help. " Do you mean to say have a fully-connected layer after the conv layer or at the end i.e. before softmax and classification layer?
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training