How do I repeat same results by using the same parameters and sample when training neural network (like BP) in MATLAB? For example, I want to compare the effects of different nodes of hidden layer on neural network training. But even if I set the same initial weights and bias, same sample and nodes the trained neural network capabilities are still different each time. What causes the phenomena? Is the algorithm inherently random (e.g. gradient descent)?
This is my code.W1 W2 B1 B2 is unchanged。
net = feedforwardnet(S1);
net.InputWeights{1}.userdata=W1;
net.biases {1} .userdata=B1;
net.layerWeights{1}.userdata=W2;
net.biases {2} .userdata=B2;
view(net);
[net,tr]= train(net,P,T);
Thank you!
In MATLAB the function rng is used to control random number generation.
Try to use the following statement at the beginning of your code for reproducibility of results.
seedRandom = rng(1,'twister');
Use rng(seed) to keep any random process in your code fairly the same each time you run the code
Hello. Your question is a little unclear. Are you using back propagation with gradient descent (GD) or with stochastic gradient descent (SGD)? If you are training a deep convolutional neural network (convnet) using image data, you are probably using SGD. It is a requirement for the correct operation of SGD that the subset of images selected from the training data set (i.e. the minibatch) be randomised at each iteration of the backpropagation (BP) algorithm. Apart from this, there is no other source of randomness in an actual training run (as far as I'm aware). The sampling operation is usually exclusive (without replacement) until the whole training set has been sampled, which constitutes one training epoch. This process is then repeated.
If you have followed the other respondents' advice, you will be fixing the pseudo-random number generator seed to the same value. The training should proceed in exactly the same way at each run, assuming you are restarting the application rather than continuing from the same instance of the application and therefore using a different set of random numbers for the SGD minibatch selection.
The only other thing I can think of is that you are perhaps stopping the training at a different point each time. The weight and bias parameters of the network would normally be different for different numbers of epochs.
Please recommend this answer if you find it useful.
- Badges
- Science topic