Hi Guys, is there any current research done as to figuring out a way to reduce the size of DenseNet for Image classification ? This can either be through weight quantization or removing of skip connections which don't add too much value to the final classification task.
Hi Titas,
You will find various versions of densenets with varying depths (sizes) from the original's paper GitHub repository. (https://github.com/liuzhuang13/DenseNet)
Or are you looking for something else?
Hello!
1) Usual Dense Layers is not so practical as Convolutional Neural Networks which created to reduce the size of neural network for image data.
2) Neural network quantization
You could look at quantized neural network (neural network quantization).
It is used to reduce the size of neural network, for example, for mobile devices, where the size of system is critical.
Main introduction:
https://jackwish.net/neural-network-quantization-introduction.html
The interesting working ideas here: if you turn the 32-bit float values of weights and activations into (1-bit, 2-bit, 4 bit) integers in many cases the result will not change too seriously.
There are many articles and well-known ways to reduce the volume without reducing the number of nodes (using quantization approach).
3) Deep compression
The survey: Yu Cheng, Duo Wang, Pan Zhou, Tao Zhang. A Survey of Model Compression and Acceleration for Deep Neural Networks https://arxiv.org/abs/1710.09282
- Parameter pruning and sharing,
- Low-rank factorization,
- Transferred/compact convolutional filters
- Knowledge distillation.
Good luck!
Why don't you try to use feature engineering (through PCA or automated feature selection scheme etc) first and later you can use artificial neural network.
Do you want to reduce the size of an existing network or create a smaller, more efficient network. If it's the latter option, you might want to check out EfficientNet:
Preprint EfficientNet: Rethinking Model Scaling for Convolutional Neu...
If it has to be DenseNet - Separable Convolutions are a simple way to massively reduce parameter. Simply replace each convolutional layer with depthwise separable convolutions.
http://openaccess.thecvf.com/content_cvpr_2017/html/Chollet_Xception_Deep_Learning_CVPR_2017_paper.html
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification