Hi Jacky,

You could check the "Correlation coefficient" mertic. Also you can exsmine both "Mean absolute error" and "Root relative squared error". You can find them in the literature, just use google.

Enhancing the performance is a critical issue as it depends not only on ANN, but also on the data. In regards to optimise ANN, you could try modifying the number of hidden units in the hidden layer and/or the regularisation parameter.

HTH.

Samer 

Hi Jacky,

There are effectively plenty of ANN training algorithms (i.e., learning rules to adapt ANN parameters: steepest gradient rule, conjugate gradient rule, natural gradient rule , newton rule, quasi-newton rule, BFGS rule, etc.). For a given optimization criterion, these rules often differ in terms of convergence speed. Neverthless, the quality and the accuracy of one ANN learning are strongly dependent on two factors: the Empirical Risk (optimization criterion) and the Confidence Interval fixed based on the VC dimension of the learning function set. Follow this link for more details: https://www.researchgate.net/post/Any_rule_of_thumb_in_training_RNN_model.

Hoping it will be helpfull

Hi Jacky,

That probably because you have achieved the maximum network capability, Training algorithms search for the global minima. They are different from each other by the way to find the minima. But at the end all probably will find the same minima. If you want to achieve a better minima you need to change your network architecture or to use an adaptive ANN. Please check the paper attached which could help you.  

Article Online Self-Adaptive Cellular Neural Network Architecture fo...

In order to help practicing the learning theory by Vapnik while choosing the ad-hoc ANN architecture/model, I recommend these must-have references by S. Haykin in machine learning:

• https://cours.etsmtl.ca/sys843/REFS/Books/ebook_Haykin09.pdf
• https://cdn.preterhuman.net/texts/science_and_technology/artificial_intelligence/Neural%20Networks%20-%20A%20Comprehensive%20Foundation%20-%20Simon%20Haykin.pdf

Thank you to all the contribution my friends! It's really a great help for me. Would like to ask one more question, anyone of you have idea how to write the coding to determine the accuracy (except regression) , training time and also the epochs taken.

For my model, I am having the problem that training time is very short, less than 1s and also the epochs can only be 6. I would like to make a better fitting of the data. Would like to learn from all of you and your help is highly appreciated!

Hi Jacky,

ANNs are full part of machine learning paradigm. Statistics constitute the theoretical core of the learning theory it relies on. Statistics usually require enough data to accurately estimate the implicit distribution densities of learning data.

In your case study, the epoch size is very limited. So, the global optimization/learning process of the ANN has to be very efficient. In such a context, I recommend you to estimate the learning rate very accurately by adopting a so-called line search method (refer to the following issue https://www.researchgate.net/post/How_to_determine_the_best_learning_rate_to_achieve_data_convergence ).

Moreover, the accuracy of one ANN learning is strongly dependent of its generalization ability. To evaluate the latter, additional/excedentary learning data are required the use of which is restricted to quality tests.

In situation where such data are missing, super-sampling or inpainting-like methods may be used to artificially generate some additional interpolated data.

Regards 

Thank you so much for all the great response peeps!

It really helped me a lot. 

Just would like to check with all of you, how should we analysis on the training of algorithm? Why certain training algorithm is only suitable for certain processes? Is it because of the mathematical model?

Some training algorithms are strongly specialized (for example FastICA able to learn how to discriminate some independent sources from mixed signals). Others have a wider range of use such as RBF networks which may solve clustering and regression tasks.

Both the empirical risk to optimize and the parametric model/structure uniquely characterize one ANN. Both of them characterize what the ANN is able to learn and as a consequence its range of applications.

From an engineering point of view, it is better to consider any ANN as an optimization tool combining a criterion (a function designed to efficiently search for the optimal solution of one problem) and a model (parameter set encoding the optimal solution of the problem after learning). Then the learning rule, customized from the empirical risk, may be viewed as the means to search for (in guidance direction sense) the optimal solution.

In addressing how to enhance the performance of one ANN training, you may refer to this related question on researchGate: https://www.researchgate.net/post/What_are_the_limitations_of_deep_learning#591b168c93553b13ad404174

Why is certain training algorithm only suitable for certain processes? Is it because of the mathematical model?

This depends on the way of designing each scheme. Therefore, you need to prepare your data and select the suitable learning algorithm to handle it. 

HTH.

Samer