Aamir ,

What are the parameters of the neural network. Assuming it is BP:

1) are you doing batch or online?

2) what is the learning rate?

3) what is the number of epochs?

4) Do you have a stopping criteria?

If you can provide the data that you are passing to the NN and the encoding it will be helpful.

Data pre-process also need to be consider in order to achieve very good results in training.

@Arturo Geigel

3 layers, 6 input neurons, 3 hidden, 1 output neuron

1) batch

2) i am using another technique for learning having many other parameters

3)1000

4) stopping criteria is mse

I think your problem lies in the error calculation. You cannot use the absolute value of the error. Remember that when you do BP you are using the derivative of the error function when you apply the derivative the square of the error in your code disappears. This leaves you with the generalized delta rule. With batch training the trick is that the derivative of the sum is equal to the sum of the derivatives. You use the sum of the derivatives for your batch training. If you use the absolute value of the error in your BP code it will guide the error minimization in just one direction.

What I think is happening with your code is that you are using the error calculation (error = abs(original(k) - calculated(k)) ;)in your generalized delta rule modification and this messes the direction of the derivative in the concave surface.

Let me know if this helps with your code. If not let me know so that we can continue the troubleshooting.

Regards

Thnx. your discussion has helped me much but still my problem remains as i donot have full command on that technique ANN.

Following some info i also want u to know

1)I am using another technique for updating weights that include gradient for searching in vector space in the right direction so how can i then use BP formula as i am using MSE and other technique for updating weights. can u help me wht type of technique should be used there which can update weight successfully for convergence?

2) I am bit confused about mse. for example, for 1st training pattern, i am updating waits in each epoch/iteration till mse is achieved but problem is that wht n, samples, value should i use for 1st pattern and for last pattern? should i use n value as processed samples till then or total samples value in every mse calculation?

3) I m doing as taking inputs and outputs for 1 case and update it till my measured and target out matches with lowest error as describe by mse

Kindly guide me if i am going in the right direction as i m not getting any guidance from any one around me.

@Arturo Geigel

I am waiting for your response. Plz do respond soon.

Sorry for the delay I did not see your response earlier. You can also send me a direct communication i you see that I do not respond.

With regards to your post I will try to address them one by one

1) The basic problem is that even if you search for the correct vector direction(using second order methods for example) will not guarantee convergence to a local minima due to the nature of BP. I would suggest stochastic methods to perturb your solution to see if you are in a local minima.

2) There are two basic training methods for BP online and batch. With online training you take one sample, compute the error and backpropagate (this is the formula derived on most books. For batch training you calculate the error for each sample and add the error. when you have presented all samples, then you do the backpropagation based on the cumulative error. You can use the error as is or a normalized equation to do it. You can do a variation on batch learning with n samples(which I think is the core question you are asking) to save computational time but I would test with all three methods to see time vs accuracy trade off.

3) you should train on your complete dataset before presenting your samples again( I would suggest randomizing the training set at each epoch so that the same overall pattern is avoided). I would also suggest early stopping or extended early

stopping

Hope this helps

you can easily perform it using matlab. please read matlab help for writing ANN code. if you want, i can send you its simple code. instead of BP, you can choose LM algorithm to train ANN. you must select ANN input accurately and all affecting parameters must be considered. also, it is necessary to convert you input-output data in the range of [-1 1]. this technique help you to earn better results. also MSE must be reasonable. it can be 1e-2 or 1e-6 ! it depends to your system. please select various MSE to find best one. also, you must test various number of hidden layers and neurons to find better convergence.