I did Implemented Error back Propagation training algorithm in that I have 52 training pairs, 1 input node, 2 hidden node and 1 output node by using these configuration I train neural network, but it will take 24 hours for training after that, it can not minimize error signal. Please tell me what can I do? And give valuable suggestions.
Dear Rahul
There are many things that could impact learning time. Assuming that your code is ok I suggest to check the following things:
1) If is a classification problem, it may not converge if the clases are non separable
2) If you did not normalize your date, specially the output there are high chances that the net is not going to converge. Normalization depends on the neuron activation function as well as if is a regression or classification problem. Remember that you should avoid big errors in the output layer. You should be aware that errors should be lesser than the maximum expected error which is the range of the output.
3) If your data has high dimmension, try doing a PCA for variable reduction.
4) I stongly recommned to use some other implementation in order to have a reference for your code. For instance the Matlan toolbox is one of the best. Or you can use the free available one in R (www.r-project.org).
Hope this can helps
What kind of error minimization method you use?
gradient descent? Levenberg–Marquardt algorithm? etc...
depending on the method you use the computing time can be exponential and sometimes do not converge
I think somethings are wrong.It's not logical.
write your code here please.
In the first instance, I suggest you experiment with learning rate and moment before getting into your serious experiments (i.e. with the "default" dataset you have) to establish the best initial values.
Then try scale transformations and encoding on the attributes to see if you can improve the seperability of the convex hulls.
If you find a promising line to follow, you may want to re-run an experiment with that form of encoding to optimise learning rate and momentum before proceding on a larger scale.
And one question:
what is the dimensionality of your data. with very high dimension, due to the curse of dimensionality, you may not converge.
Dear Rahul
There are many things that could impact learning time. Assuming that your code is ok I suggest to check the following things:
1) If is a classification problem, it may not converge if the clases are non separable
2) If you did not normalize your date, specially the output there are high chances that the net is not going to converge. Normalization depends on the neuron activation function as well as if is a regression or classification problem. Remember that you should avoid big errors in the output layer. You should be aware that errors should be lesser than the maximum expected error which is the range of the output.
3) If your data has high dimmension, try doing a PCA for variable reduction.
4) I stongly recommned to use some other implementation in order to have a reference for your code. For instance the Matlan toolbox is one of the best. Or you can use the free available one in R (www.r-project.org).
Hope this can helps
I woful agree with many of my colleagues above. But I would also ask that you consider the language yu are using and if you code is compiled (C++) or interpreted (Java, Python). If you are using Java or Python there are simple ways to speed things up by using appropriate matrix algebra library calls versus your own while-do (for) loops - the called libraries will use compiled binary or byte code code. This type of analysis and correction will have a major impact on any algorithm.
If you use languages like Matlab/Octave for your implementation, you could use vectorization for nearly all of your machine learning algorithms. Normalization would also help if your data features spans different ranges. This would speed up your runtime considerably than using usual sequential executions. This is assuming your data does converge.
If you want to know in detail how, you might want to check out Andrew Ng's course on Machine Learning in Coursera. The course also gives valuable insights on data convergence if that is your real problem.
In my experience there are some reasons for the bad convergence:
1.Data not normalized
2.Huge amount of data with difference in their intervals. For example a lot of data positive, and few negative
3. Nor right function, such as sigmoid. Try to change it
4. Not right merit function, try to change it.
Hope this helps...
The choice of optimization method (Stochastic Gradient Descent, momentum, rprop....) makes a difference. This visualization i find quite illuminating:
http://imgur.com/a/Hqolp#2dKCQHh
I suggest to use features rather than raw data to reduce the dimensionality, you may use statistical or shape features or combination of both. A good paper on this is "Improved SPC chart pattern recognition using statistical features"
DOI: 10.1080/0020754021000049844
The next suggestion is to change the training algorithm
Dear Rahul,
Use faster learning algorithms such as Levenberg–Marquardt one. The training time should not exceed a few seconds.
Regards
Dear Rahul
There are many things that could impact learning time. Assuming that your code is ok I suggest to check the following things:
1) In most ANN training ,one hidden layer enough to train the network,so the time of training will minimize at least to half
2)You can decrease the number of output node by using Binary Coding Mechanism( BCM) so when we decrease the number of output node the time already decreasing
Hope this can helps
Hi,
I am not sure how you are evaluating the fitness of the network. Have you considered to test which network are likely to be worth executing for the fitness or not. If they are not, just add a penalty to them.
Patricia
Hi Rahul i will suggest
1) Normalize you data.
2) Try to change no. of nodes in hidden layers.
3) Try to change activation function.
If all of this is not working try different training algorithm.
Hi Rahul,
The feedforward backpropagation learning algorithm
with steepest gradient descent can be employed for training
the network. The network parameters, like learning rate
and training epochs can be investigated in order to improve
the prediction performance, but also to assess the necessary training time. I found that for a chosen architecture and database of the network,
the learning rate variation does not affect the accuracy of
simulations, but affects the training time. You can try to play with different learning rates and number of epochs, to see whether it makes any difference.
Good luck with your work!
Luiza
Thanking to all for your valuable suggestions. I will correcting all the mistakes.
Hi,
To reduce time, you could look for possible features for success and some for failures. It works well in hyper-heuristics, where algorithms can be generated from a for of GP. The algorithm solutions can be easily scanned and only algorithms with specific operators are tested. This type of rewards relies on a human-knowledge of the problem domain.
Patricia
- Badges
- Science topic