In a convolution neural network, given an input signal, each processing element gives out an output that is fed in other elements of the network. What happens with the original signal? As it propagates through the network and splits in different pathways it passes through various elements. Is it possible that is passes through an element and continue its pathway unmodulated, regardless the output of this particular element? What does the modulation (if present) depend on?
Excuse my lack of knowledge of the technical language, as I am not an expert in computer science.
The signal is basically a feature vector, a tabular list with features. In case of an image it is typically a pixel array. As vector passes through the layers, the signal gets modified so the image transforms to higher abstraction level. It is possible that during training some neurons remain unchanged or even become unreachable so it won't modify the signal.
Imagine each neuron has an input-output function (a.k.a activation/transfer/response function) by which the input signal temporally modulates (i.e., convolutes in Fourier domain).
The neurons in a usual 'convoluted network' use nonlinear-though very simple-transfer functions (e.g., Sigmoid, Heaviside step, etc), therefore:
- If a signal get passed a neuron it is most probably modulated unless the neuron's transfer function was a constant number (e.g., a buffer element).
- You cannot decompose the original signal from the modulated signal because of the non-linear modulation.
Finally, the current convolution neural network trend has a strong computer science flavor so people don't think of its comprising neurons as signal modulators but rather as Perceptrons.
In the convolution layer only multiplication of kernel values with signal values takes place and the values and summation takes place. The fully connected layers will have a lot of connections and all possible permutation and combination of the data is available for the neuron to study and perform analysis.
- Badges
- Science topic