Are there papers that talk about the possible advantages and disadvantages of using autoencoders instead of CNNs for feature extraction in neural network?
Hi!
When using Auto-Encoders it gives us an advantage by reducing the dimensionality of the data we are using, as well as the learning time for your cases. Another very good thing is the compactness and speed in coding using backpropagation.
In an Auto-Encoders it is interesting the static topics of abstract topics so it brings a better data transition than training CNNs. Other points that can add you are: Auto-Encoders can be trained in an unlabeled way; Auto-Encoders are unsupervised models and CNNs already need them;
Below are articles about:
- https://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma
- https://www.researchgate.net/publication/301632899_An_overview_of_Convolutional_and_AutoEncoder_Deep_Learning_Algorithm
- https://people.idsia.ch/~juergen/icann2011stack.pdf
Autoencoders can be great for feature extraction. A purely linear autoencoder, if it converges to the global optima, will actually converge to the PCA representation of your data. In that sense, autoencoders are used for feature extraction far more than people realize.
You asked for disadvantages, so I'll focus on that.
You have to train an autoencoder. That's a lot of data, processing time, hyperparameter tuning, and model validation before you even start building the real model.
An autoencoder learns to efficiently represent a manifold on which the training data lies. If your training data is not representative of your testing data, then you wind up obscuring information rather than clarifying it. I'm working on such a problem right now. I wanted to use an autoencoder but, it would actually decrease performance.
An autoencoder learns to capture as much information as possible rather than as much relevant information as possible. That means that if the information most relevant to your problem makes up only a small (in magnitude) part of the input, the autoencoder may lose a lot of it.
Ref : from quora - 📷Conner Davis, Data Scientist at MicrosoftAnswered March 11, 2017·Upvoted byDrew Levin, PhD Computer Science & Complex Adaptive Systems, University of New MexicoandAkshay Khatri, MS Computer Science, Columbia University (2018)
Hello
https://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma
may be useful for you
- Badges
- Science topic
- Similar topics
- Papers