Hey,
as far as I know, batch size and iteration are in the end leading to your epoch. What does this mean?
Batch size is the size of your batch - obviously - you are giving your network to work and learn with. Your are usually doing this, because of the limitations of your PC, such that you are unable to learn on ALL date at once. Iterations on the other hand are - obviously again - your iterations, so: how often do you iterate over your data AND your batches. Epoch now means a time interval in which your network has seen and learned every data point once, meaning you should set your iterations in dependency of your batch size such that your network has seen every datapoint for a reasonable amount of times.
Hope this helps (and is correct).
André
Hi Vibhor Kumar Vishnoi ,
suppose that you have 100 training examples. If you set the batch size to 20, you will have a total of 5 batches. Then, you start the training of your network with one batch until a convergence criteron or the maximal number of iterations is achieved. When you have finished the training with all the 5 batches, you have completed one epoch. Why multiple epochs are often used? Because the sequence in which the data see the network can affect the calculation of the optimal weights. So a better generalization can be obtained if you use multiple epochs to avoid such problem. Of course one should shuffle the data before starting a new epoch.
I hope this helps you.
Felix.
Batch size - number of training examples used to compute ONE gradient step (backward pass) of your network by averaging them.
Iteration - one update of network's weights (forward pass of Batch size of data and ONE backward pass).
Epoch - when the network saw all the data in your training set, i.e. when Iteration = Number of training data / Batch size
- Badges
- Science topic