Concatenation or combination is a new approach in deep learning. it increases the precision of learning and the discovery of a new architecture. the question that arises and what are the operations used to combine between the features (is it a sum, average, average weight, svm, Min/Max, etc) and what is the best methods.
Djallal RAFIK Hammou ,Generally CNN architectures have Convolutional layer in which dot product is used. Also, in Pooling layer, either max pooling or average pooling (i.e. max or average operation) is used.
A concatenation operation is just a stacking operation. For example, the x1 layer has 256 channels, and the x2 layer has 256 channels. If we are concatenating these two layers channel-wise then the output of concatenation will have 512 channels.
On the resulting tensor, we can perform all the available operations as to any other tensor.
The improvement in results due to concatenation is may be due to the fact that concatenation operation increases the features space by combining the high-level and low-level features. Thus the subsequent Convolutional operation is able to learn new features that are dependent on both high-level as well as low-level features.
I hope this answers your query.
To understand the philosophy or role of concatenation in data analysis, I suggest you to follow https://www.annielytics.com/blog/excel-tips/17-real-world-examles-of-concatenating-marketing-data-in-excel/
- Badges
- Science topic
- Similar topics
- Design
- Architecture