This maybe is because your neural network is initialised with random values of gains or weights, then you'd have different starting points per each simulation during the training phase. To have always the same result, try to propose a fixed seed for the initial weights. Another option is that the neural network works with a random data division, taking randomly the order of input/target pairs to compute the error propagation phase. Try to check if your issue is coming by one of the two above possibilities or even if it is coming from both. 

One idea is to run the neural network as you have done until now and then take the configuration with better performance.

This maybe is because your neural network is initialised with random values of gains or weights, then you'd have different starting points per each simulation during the training phase. To have always the same result, try to propose a fixed seed for the initial weights. Another option is that the neural network works with a random data division, taking randomly the order of input/target pairs to compute the error propagation phase. Try to check if your issue is coming by one of the two above possibilities or even if it is coming from both. 

One idea is to run the neural network as you have done until now and then take the configuration with better performance.

Naturally, this is because random weight assignment in the first iteration. However, a stabilized network  is good enough to model Input output mapping.

It is good indicator, that control parameter values of your NN are not selected well. After good tuning the difference (due to difference of the init random generator values) should be negligible. 

The objective is to find the weights and biases for which the performance function is within a stated tolerance of a global minimum on the test set. The performance surface can be very convoluted and have many local minima. So the final weights and biases can vary with each training session. Usually, the weights and biases are initiated randomly. The random number generator can be set to initiate the same wights and biases each time. Better is to find a suitable architecture and then save the weights and biases. These can be used as the initial weights for further training. 

Dear Thomas,

Like many people already pointed out (especially Mike), it is not unusual to obtain slightly different results after training ANNs as model parameters (weights) are randomly initialized at the start of training. However, an early stopping or cross-validation scheme may help in building more robust models. i.e. more stable generalization performance.

As stated by others, the random initial weights will lead to different solutions each time you train the neural network. If you want the results to be the same each time, for comparison and reproducibility, you can set the initial weights to the same values each time. This can be achieved by using the same value to seed the random number generator each time you run your program.

If you are using ANN available in MATLAB than before training the ANN each time type the following in command window and enter;

default rng;

For training ANN using PSO in MATLAB which gives better performance that the MATLAB training algorithms, see the following article;

https://www.researchgate.net/publication/305325563_Codes_in_MATLAB_for_Training_Artificial_Neural_Network_using_Particle_Swarm_Optimization

Code Codes in MATLAB for Training Artificial Neural Network using...

Though it is usually the result pattern, especially with multilayer perception neural networks, but there are some algorithms like the generalized regression neural networks and radial basis function neural networks whose results are stable. So, you can try these two networks.

You need to set your random number generation fixed, so called seed of random number generation

.

usually first stabilize the neural network and then try it, moreover instead of using random inputs, better try with unique set of input values. your problem may be resolved.

Use the line below in your mfile before training the network:

rng('default');

It will make sure to use same starting weight and bias value sets and as a result the network ends up trained exactly same each time.

As rightly pointed out in previous answers, it may be due to the varying network weights that get randomly initialized each time you create the network.

If, however, you want to observe the effect on network generalization by variations in other network parameter, then you may fix these network weights, by creating them once (and save in a file) and then mapping them onto the network each time before training.

In Matlab if 'net' is your neural network variable then

net.IW refers to input weights and net.LW refers to (Hidden)Layer weights. The values of these may be set to a valid weight matrix of the same size. e.g.

InputlayerWtmatrix = (input x hidden) size wt matrix for input-hidden layer

hiddenlayerWtmatrix = (hidden x output) size wt matrix for hidden-output layer

...........

...........

net.IW{1,1} = InputlayerWtmatrix;

net.LW{2,1} = hiddenlayerWtmatrix;

net.b{1,1} = B1; % B1 is the input layer bias matrix

net.b{2,1} = B2; % B2 is the hidden layer bias matrix

@ Md. Shafiqul Islam What about in python

@Manu Ratheesh in python (scikit-learn) you should set random_state=N when you create the network, where N is an integer

Md. Shafiqul Islam i did it but there is an error when run mfile:

"First input must be a nonnegative integer seed less than 2^32, 'shuffle', 'default', or generator settings captured previously using S = RNG.

Error in CONC (line 75)

rng('defult')" .

any idea?

Hi Seyyed sajjad Rezaeinasab

I am sorry that I could not understand the error message fully without the code. If you by any chance are using Matlab released before 2009, please read the webpage:

https://se.mathworks.com/help/matlab/examples/controlling-random-number-generation.html

Cheers!

Shafiq

• Thomas lass Barna Hi, here is the solution. first at the end of your ANN mfile type: s=rng then run it. after that before beginning the train commands type: rng default in your mfile. that's it. hereafter you will have the same answer in every run.

here I developed matlab code for an simple volterra integral equation, but I could not able to figure out,

the integral equation and code, can some please refine this code, My intention is the error value I could able to get upto 10^7, I am expecting the error refinement upto 10^14. Can someone help me in this regards.