We know that a neural network is by default fully connected. But recently i found in a paper, "Deep Learning" by LeCun, Bengio and Hinton where some of the figures of neural networks are partially connected. How far it is genuine ?.
Ideally, a Neural Network ought to be comprised of layers containing fully-connected MLP units (neurons). However, Neural Networks can still exist and function with layers comprising partially-connected neurons. Thus, having neural layers with partially-connected neurons may incur unnecessary computational resources than required. Hence, the need for neural pruning (i.e. dropping out of less relevant neurons) with respect to the neural architecture.
...the authors concluded that the performance of the network can, in general, increase when using partial connectivity with a lower...
Check this article: (http://www.korczak-leliwa.pl/files/papers/9-Elizondo-Fiesler-Korczak.pdf)
Yes. It is called a partially connected neural network (PCNN), a network which contains only a sub-set of the entire set of possible connections for a particular neural network model. You have to know the nodes you are going to connect.
Yes, definitely. Most popular Convolutional neural netwok commonly used for image classification is a type of partially connected network. Partially connecting it helps in reducing the number of parameters to learn so it also decrease the time complexity of the algorithm
I think the answer provided by Dr.Javier by adding a review article in the way puts an end to my query. No need for any further clarification/answer.
Yes. In fact, we use a specific operation, namely dropout, that removes some connections between neurons to introduce noise to the network and avoid overfitting by doing so.
- Badges
- Science topic
- Similar topics
- Biological Science
- Bioecology
- Systems Ecology
- Landscape Ecology
- Connectivity