Transfer Learning Vs. Designing CNN cons and pros
https://www.linkedin.com/pulse/transfer-learning-vs-designing-cnn-cons-pros-dr-wafaa-shalash
let us exchange our experience of comparing transfer learning and designing deep neural from scratch.
No doubt that deep learning, especially convolution neural networks (CNN) shows excellent success in the field of pattern recognition and classifications. When you decide to use the deep learning technique in your application, you either have to choose one of two strategies. These two strategies are designing and training your own CNN from scratch or reusing a retrained net and adapt it to suit your application. Designing your CNN from scratch is not an easy solution. It needs a lot of effort and time to construct a suitable combination of layers and adjust your designed CNN hyper-parameters.
Moreover, you need a sufficient amount of data (maybe a few thousand), and also you need high computational power (at least a high-performance GPU). On the other hand, if you choose transfer learning you have just to adjust input and output layers according to your problem and retrain either the hole network or just the fully connected layers. Transfer learning seems to a preferable solution, especially with lacking of data and GPU. But regarding the final result, if you succeeded in designing and train your CNN it will give huger accuracy than the transfer learning solution.
I emphasized this while developing a solution to driver fatigue detection using EEG signal using the same datasets and the same input representation through two of my research papers. The first is: "W. M. Shalash, "Driver Fatigue Detection with Single EEG Channel Using Transfer Learning," 2019 IEEE International Conference on Imaging Systems and Techniques (IST), 2019, pp. 1-6, doi: 10.1109/IST48021.2019.9010483."
and the second is: "W. M. Shalash, ‘A DEEP LEARNING CNN MODEL FOR DRIVER FATIGUE DETECTION USING SINGLE EEG CHANNEL’, J. Theor. Appl. Inf. Technol., vol. 99, no. 2, pp. 462–477, 2021".
For CNN you need to more preprocessing of the dataset but with transfer learning you only need to little processing of dataset like resize to 227 x 227 or 224 x224 according to selected Pre-trained Models (AlexNet, GoogLeNet, ResNet, VGG Networks etc. and more) this saves much time of preprocessing data.
Pretrained Models gives much greater accuracy for completely new dataset such as image dataset.
As I use transfer learning for Handwritten Character recognition gives higher accuracy. While, CNN gives slightly lesser accuracy than transfer learning. When I have used single CPU which itself takes lot of time in training and one of the model is also failed to train the dataset. Thus parallel computing should preferred in transfer learning of training data.
Advantages of Transfer Learning
- Transfer learning is useful for insufficient data and imbalanced class problem. So that we can limit training data and training time. It can give nearly hundred percent of accuracy for less amount of data.
- Transfer learning gives good combination of features even for very complex task within short time frame.
disadvantages of CNN
· Training a convolutional neural network takes a long time, especially with huge datasets.
· While CNNs are translation-invariant, they struggle with rotation and scale-invariance unless data is augmented explicitly.
Dear Wafaa Mohib Shalash
These links might seems to connect all the dots together, have a look:
1. https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a
2. Article Review of deep learning: concepts, CNN architectures, challe...
Kind Regards
Qamar Ul Islam
Thanks all of you for sharing your experience hope to see more to enrich our experience
- Badges
- Science topic