The ResNet(Residual Network) was introduced after CNN (Convolutional Neural Network). Additional layers are added to a DNN to improve accuracy and performance and are useful in solving complex problems. The intuition was that these layers would progressively learn the features. But it has been found that there is a maximum threshold for depth with the traditional Convolutional neural network model. That is with adding more layers on top of a network, its performance degrades. This problem of training very deep networks has been alleviated with the introduction of ResNet or residual networks.
My query is can we see ResNet as special form of CNN? Then how?
Any form of CNN can be considered special. Any blog or paper which describes the various types of CNNs will let one think that those are the special types of CNNs.
To answer your questions, yes and no. It depends on the context of your answer. Considering a basic form of CNN such as LeNet as a "Normal" CNN can make ResNet a special CNN. But judging LeNet as a baseline architecture won't really do so. ResNet or any other model can be termed as an "improvement".
Also, could you please send a link to the proof of your statement - "it has been found that there is a maximum threshold for depth with the traditional Convolutional neural network model"?
Yes.
Earlier deep CNN architectures were hard/impossible to train due to the nature of backpropagation algorithm (vanishing/exploding gradients).
Resnet added skip connections that allowed for better application of the backpropagation training algorithm.
Yes. ResNet is a way to handle the vanishing gradient problem in very deep CNNs. They work by skipping some layers assuming the fact that very deep networks should not produce a training error higher than its shallower counterparts. In an overall perspective they can be thought of as a model similar to LSTM in RNNs.
- Badges
- Science topic