Hello
I want to know if there are ways to know the minimum amount of training data for a neural network to get good performance.
I have a histogram of 73 features to be feed to neural network of 73 inputs and one output with hidden layer and 69 training samples, but I think this is not enough.
What do you think about that ?!
Dear Salah Eddine
According with your comment an increase of training sampling will allow get better results. Also considering your experiment I recommend a review of information about sampling theory from statistics point of view.
Best regards,
D. Escobedo
I would recommend you run a few tests to see what is the minimum number of trials it takes the network to converge to a satisfactory solution. In theory, there is a global minimum of error, but it is unlikely one will ever find it. One will, however, be able to approximate it and settle on a successful solution after a certain amount of trials. After a training, you should test your network, e.g. if you present the training set 10000 times, that would equal 10000 trials. After 10000 trials it might converge to a nice solution, and you might even get a nice solution with less training trials. With the processing power of modern PCs or workstations, the training should be completed very fast (e.g. 10000 presentations of the training material does not take your computer or workstation very long to calculate). The speed of training also depends on your number of hidden units (the more hidden units you have, the more connections there are, the longer the training takes). I personally chose the minimum number of trials and the minimum number of hidden units it takes the network to converge on the solution.
A good approach to problem-solving is to first try a problem that is related but simpler:
First, consider how many parameters your network must learn in the simplest possible type of network:
Your network takes a 73-element vector (a histogram) as input, and produces one output. Let's presume that your training data, has one target value representing the desired output that the network is supposed to produce at each step.
In a recurrent neural network with one output, you would also have at least one extra "input," which is the network output from the previous input vector: so effectively, your recurrent network has 74 inputs. Furthermore, if it has a bias term fixed to 1.0, that can also be considered an input, making a total of 75 network inputs (but you still only need to supply the 73-element vector).
"Learning" is the problem of assigning network weights that map the input vector to the output. You must determine a minimum of either 74 or 75 weight parameters.
The simplest possible way to determine weights would be a linear combination, which can be solved by regression (linear and logistic regression will be affected by the same issues).
Now ask yourself, if you are simply trying to calculate a linear mapping of 73-element input vectors into a scalar output using linear regression, do problems arise when your data set consists of 69 training samples?
This problem does not have any specific answer. But training data is 70% of total data 15% for testing and 15 % for validation.
- Badges
- Science topic