I am using autoencoder networks for deep learning based feature extraction purpose for spectral images. For the time being, the number of hidden nodes in the network is randomly chosen.
Is there any way to optimize this parameter so that the best feature representation is achieved?
There are some practical ways to determine the best size, but it all comes to evaluations. The general rule is the optimal size of the hidden layer is usually between the size of the input and size of the output layers.
In sum, for most problems, one could probably get decent performance (even without a second optimization step) by setting the hidden layer configuration using just two rules: (i) number of hidden layers equals one; and (ii) the number of neurons in that layer is the mean of the neurons in the input and output layers.
Jeff Heaton, author of Introduction to Neural Networks in Java offers a few more.
There are some practical ways to determine the best size, but it all comes to evaluations. The general rule is the optimal size of the hidden layer is usually between the size of the input and size of the output layers.
In sum, for most problems, one could probably get decent performance (even without a second optimization step) by setting the hidden layer configuration using just two rules: (i) number of hidden layers equals one; and (ii) the number of neurons in that layer is the mean of the neurons in the input and output layers.
Jeff Heaton, author of Introduction to Neural Networks in Java offers a few more.
Issa Annamoradnejad What if the number of nodes in the input and output are the same? Usually in autoencoders, the number of neurons in the input layer and the output layer are same.
Pangambam Sendash Singh Well, I suppose it should be equal to the same number; but I'm not so sure. You should decrease and increase (a small change in the order of 1-10%) to see if it changes positively.
if you use linear encoder phase and linear decoder phase with squared error loss function, then under that condition the autoencoder acts as PCA.
For configuring the size of hidden code, you have to code encoder and decoder separately, and then alter the code_size variable as in this example:
def build_autoencoder(img_shape, code_size):
# The encoder
encoder = Sequential()
encoder.add(InputLayer(img_shape))
encoder.add(Flatten())
encoder.add(Dense(code_size))
# The decoder
decoder = Sequential()
decoder.add(InputLayer((code_size,)))
decoder.add(Dense(np.prod(img_shape)))
decoder.add(Reshape(img_shape))
return encoder, decoder
Mohammad Khalid Pandit Thanks for the reply to my question. By the way, my question is about how to determine the code_size for my problem.
Have a look at https://bradleyboehmke.github.io/HOML/autoencoders.html
- Badges
- Science topic