Dear all,
I am applying ANN method in studying biodiesel production. This was performed by using the neural network toolbox of MATLAB R2015a. I select Feed-forward neural network (FFNN) with back-propagation training (BP), BP algorithm is based on the LM training function. The TANSIG transfer function was selected for the input and output layers. After finished training and repeat some runs in the same condition, I realized that the results about MSE, R^2 were not same. Why? I think that the reason for this difference is in the total of experimental data points, and then it was randomly divided into three subsets such as training set (70% of total), testing set (15% of total), and validation set (15% of total). This random division can lead to the different values about MSE and R^2 when we repeat.
So how do you think about this and how to optimize this training in ANN?
I was also known that optimum training of the ANN can be obtained through particle swarm optimization (PSO). And how I can apply this algorithm?
Thanks so much for your interests.
Dear Dang Nguyen Thoai,
I suggest to you a links and attached files in topics.
-Prediction of Cold Flow Properties of Biodiesel Fuel Using Artificial ...
www.sciencedirect.com/science/.../S18770509163258...
-Response surface optimization and artificial neural network modeling ...
pubs.rsc.org/-/content/articlehtml/2015/ra/c5ra11911a
-Application of modelling techniques for prediction and optimization of ...
www.academia.edu/.../Application_of_modelling_techniques_for_...
Best regards
Dear Dang Nguyen Thoai, I had the same problem that you said when i used BP nerual network for predicating the drying process of sewage sludge. I think one is that the division process for training set, testing set and validation set is random, and another is that initial weights and thresholds are random. If you repeat the whole process, the resluts must be different. You can save the trained net to solve this problem. Good luck for you.
Dear Dang Nguyen Thoai,
In the first place, each time when you run the ANN algorithm, there's a different training/valid/test data separation since they are randomly assigned to this three sets. This accounts for the different results you've observed.
Secondly, for ANN, the model performance is, to a large extent, dominated by the number of hidden layer neurons and the training algorithm you choose. As a result, a easy way to improve the performance of your application is to tuning this two hyper-parameters. To do this, you may use the cross-validation method.
Finally, since ANN couldn't always guarantee a global minimum solution, using bio-inspired algorithms such as PSO may help improve the performance. But that doesn't always works since a high training performance may cause overfitting problems. To do this, a ANN model or its weights could be defined as an individual, and the performance on test set could be defined as the fitness function. Then just let the algorithms to choose a "global solution" for you. There're many relevant papers on this issue, some of them even uses multi-objective methods.
Hope my answers are helpful, Good Luck!
- Badges
- Science topic