I usually freeze the feature extractor and unfreeze the classifier or last two/three layers. It depends on your dataset, if you have enough data and computation power you can unfreeze more layers and retrain the model.
A clue to this issue can is resolved in the attached useful links: https://www.aclweb.org/anthology/N19-1213.pdf
and
https://forums.fast.ai/t/why-do-we-need-to-unfreeze-the-learner-everytime-before-retarining-even-if-learn-fit-one-cycle-works-fine-without-learn-unfreeze/41614
very much slow without GPU, wanted to limit the data set size choose the images of interested and freeze the feature extractor to the size , once model is created with classifier can be stop so that achieved with learning rate.
It helps to know the architecture of the pre-trained model, so you know which feature-maps to use and which to retrain. Generally the efficacy of utilizing a pre-trained model is that you don't have to start from the beginning. Many CNNs, for example, terminate in a dense layer, combining feature maps to give a a result whether for classification or regression. In an encoder-decoder system, that dense layer is what shapes the encoding, so it is useful to retrain this layer for your dataset. Retraining some of the last few feature generating layers (Convolution Layers) may help in the retraining.
- Badges
- Science topic
- Similar topics
- Design
- Architecture