When we using a RBM generative model for extracting feature, for the better performance and more accurate learning features, how to portion the number of hidden units, input dimension and quantity of samples? what's the reasonable scale between those parameters? Also is it related with number of categories when we using classification RBM or MLP?
http://www.cs.toronto.edu/~larocheh/publications/icml-2008-discriminative-rbm.pdf
Oh, my friend... I must describe not that clear, you may misunderstand what I mean.
I will explain it by an example, for the RBM model, with whose input is 100(number of samples) * 1,000,000(number of input dimensions). which number of hidden units is proper for this input data? Or how do we choose the number of hidden units, based on number of samples or input dimensions? I'm just wondering which is the crucial element when we set up a RBM model?
Also, it is certainly not formal for the input of 100 * 1,000,000, for a better RBM model, we should decrease the dimensions of the input to make the data become more proper for input, and decrease the dimensions to how many should be useful and better for model, 100 or 1000?
So I just want to make the relationship between those parameters, how to choose the proper numbers of input dimensions or hidden units?
Trial, and, error.
And, cross validation or independent testing, or a really robust information based measure of model performance. I recommend minimum description length, but only because I've done that for nonlinear time series.
- Badges
- Science topic