Dear Ahsan,

Selecting the type of an algorithm is strongly related to your data set which you use. However, do not forget that normalization  could improve the performance.  

What's your specific problem, is it about a classification problem ! or regression (prediction), because the parameters of an ANN algorithm are dependents on the problem type itself. so if you estimate for exemple the state or covariance, i think that means a non-linear relationship because EKF is the nonlinear version of KF, so a simple back propagation NN is enough, and how you can said that the NN has a poor performance on testing results !

Dear friends,

Is it better to pass error in batch at the end of epoch OR 

after each error generated by NN output - desired value ?????

Thanks friends,

I think, I should break training data into batches as my data is huge. 

How can I pass error for back propagation at the end of Epoch??? is it an average of all error or RMSE??? 

you can calculate error manually in each epoch, but this take a time and the most important think in ANN training is the adjusted weights when training is terminated, so why you calculate error in each epoch !, also, when you attend the least RMSE or any other performance you take your adjusted weights and analyze your NN model if he give a logical results or not, you can also make a parametric study to show the feasibility of your NN.

Dear Friend,

Error can be one at the end of Epoch or at training. But how can you back propagate it toward the weight of input layer to first hidden layer?

WeightNew_(input layer, first hidden layer)   =  WeightOld_(input layer, first hidden layer) - ( Psi * Delta ( First hidden layer) * Input layer )

I have 2D data (100*40) , I have to give row wise training by putting 40 input given in columns.

Sum of all 100 errors will be one, but which input row I should take to generate Delta or H matrix?  

Neural network with 2 hidden layers / cost function / back propagation implemented in Matlab / vectorized implementation, i.e. much faster than standard implementations with loops 

https://github.com/gtesei/ml/blob/cab7bd445f0c7f1a2e9393e114b746d471443008/neural/turbo2NNCostFunction.m

how to invoke fmincg to minimize the cost function 

https://github.com/gtesei/ml/blob/cab7bd445f0c7f1a2e9393e114b746d471443008/neural/autoExplodeTrainAndPredict.m

Dear Ahsan,

Please see the backpropagation algorithm with gradient descent. More details in thse publications:

Demuth H, Beale M. Neural network toolbox users guide. Math-

Works, Inc.; 2003.

Veelenturf LPJ. Analysis and applications of artificial neural

networks. Prentice-Hall Intl (UK) Ltd; 1995. p. 70–6.

Good luck with your work!

Luiza

error back propagation perceptron training algorithm {EBPTA}

that is it... refer book of J M. Zurada on ANN for more detail

I did this a while back and it may be useful to you.

http://unicorninvesting.us/content/fast-compressed-neural-network-r-evaluation

Following is helpful:

https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi