Ur question is a complete course on ANN. I will recommend going through mathematical equations used in training an ANN as improving RMSE depends on many factors e.g no of training examples, their accuracy, pre/post processing, traing algo , no of neurns, hidden layers to name few. Also, the performance parameters are inbuilt in all ANN models/ architectures in MATlab as structures for which u need to see the content file and help commands.
In addition to Imran Shafi and Jacek Bartman suggested parameters you have to also consider the values of mu and learning rate. More importantly training algorithm itself also has influence on error-different algorithm work differently with a particular problem.